So, I recently had 8000+ text files. Each file contained a single line of CSV text without a trailing new-line.<\/p>\n
This 8000+ files also had subsets to them based on a unique word in the file name as well (eg: server1, serverdb etc…)<\/p>\n
My challenge was to concatenate all of the files from each group into their own individually collected files while also adding line-feeds between each file’s contents in order to create the new single CSV file and pre-pend today’s date to the file name, which would give me a proper CSV file that would be easier to share, store and process by others.<\/p>\n
First, I whip open my friendly Notepad application and create a header.csv text file that contains the column headers for the CSV.<\/p>\n
Then using Notepad again, after some unnecessarily painful searching, I finally was able to cobble together this, which I then Select All > Copy, then Paste into my DOS prompt (CD’d to the folder with the files first):<\/p>\n
type header.csv >\u00a0%date:~10,4%%date:~4,2%%date:~7,2%-SERVER1.csv && for %f in (*SERVER1*.log) do @echo. >> %date:~10,4%%date:~4,2%%date:~7,2%-SERVER1.csv && type “%f” >> %date:~10,4%%date:~4,2%%date:~7,2%-SERVER1.csv
\ntype header.csv >\u00a0%date:~10,4%%date:~4,2%%date:~7,2%-SERVERDB.csv && for %f in (*SERVERDB*.log) do @echo. >> %date:~10,4%%date:~4,2%%date:~7,2%-SERVERDB.csv && type “%f” >> %date:~10,4%%date:~4,2%%date:~7,2%-SERVERDB.csv<\/p><\/blockquote>\n(….repeat as needed… etc…)<\/p>\n
\n
- for %f in … (runs a loop of the full command-line against the wildcard based on the number of files matching the wildcard)<\/li>\n
- @echo. (with the important, trailing “.” outputs the new-line)<\/li>\n
- %date:… (captures just a portion of the “date” value and echos it)<\/li>\n
- && (appends additional command to the line)<\/li>\n
- >> (appends the output of the line to the file)<\/li>\n<\/ul>\n
Yeah, it’s crude, but I’ll take the win.<\/p>\n
If I really wanted to be fancy, I’d try and capture the date from the source files and use that for the imprint on the destination, but I’m not that inspired. Sorry.<\/p>\n
I’m posting this on the hopes that if someone spirals into this kind of situation like I did today, they’ll find this post and go “Yes! That’s exactly what I needed!”. Good luck.<\/p>\n
I’ll file this under TIL about abusing “date” and “&&” in a new way, or maybe I only learned how much DOS I’ve forgotten. Hmm…<\/p>\n","protected":false},"excerpt":{"rendered":"
So, I recently had 8000+ text files. Each file contained a single line of CSV text without a trailing new-line. This 8000+ files also had subsets to them based on a unique word in the file name as well (eg: server1, serverdb etc…) My challenge was to concatenate all of the files from each group […]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[117],"tags":[121,122,120,118,119],"class_list":["post-531","post","type-post","status-publish","format-standard","hentry","category-grody-hacks","tag-append","tag-concatination","tag-date","tag-dos","tag-echo"],"_links":{"self":[{"href":"https:\/\/www.kitchencloset.com\/home\/bryan\/blog\/wp-json\/wp\/v2\/posts\/531","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.kitchencloset.com\/home\/bryan\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.kitchencloset.com\/home\/bryan\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.kitchencloset.com\/home\/bryan\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.kitchencloset.com\/home\/bryan\/blog\/wp-json\/wp\/v2\/comments?post=531"}],"version-history":[{"count":15,"href":"https:\/\/www.kitchencloset.com\/home\/bryan\/blog\/wp-json\/wp\/v2\/posts\/531\/revisions"}],"predecessor-version":[{"id":541,"href":"https:\/\/www.kitchencloset.com\/home\/bryan\/blog\/wp-json\/wp\/v2\/posts\/531\/revisions\/541"}],"wp:attachment":[{"href":"https:\/\/www.kitchencloset.com\/home\/bryan\/blog\/wp-json\/wp\/v2\/media?parent=531"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.kitchencloset.com\/home\/bryan\/blog\/wp-json\/wp\/v2\/categories?post=531"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.kitchencloset.com\/home\/bryan\/blog\/wp-json\/wp\/v2\/tags?post=531"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}