[ale] How Pipes Saved My Bacon

[Jonathan Meek]

Hey everyone,

Hope y’all are doing well in these interesting times we live in. 

I wanted to share a thing that happen today with the hopes that it would be mildly interesting and helpful in the future. 

So set the scene, I am Java application dev who in a previous life spent time helping teams migrating out of homegrown source control tools into git. That previous role required daily interaction with bash/Perl and pipes. 

Today’s fire: Logs for a vendor product got pushed to Cloud logger but wasn’t happening on the local box (our fault, we held the product wrong and let the steam out of the side). So got the logs but it’s in the cloud format and added bonus the developers using the vendor product know one way to read the logs: how it is on the local box.

Pulled up the cloud console. 
- Can I download the logs?
   - Yes, but 10K line count limit.
- Can send the logs to a sink?
   - Yes, but can’t do it retroactively. 
- Can I look at the logs from a command line?
   - Yes and there’s no line limit!

Basic command:
‘cloud logging command’ | tee log_holder_so_dont_have_pull_again.json | jq ‘format the json to look like what the local file looks like’ > final_file.log

Took 2 hrs from “we might be answering questions on Tuesday” to “new runbook procedures in case we need them”

There’s probably a better way to do this but thought it was a cool use of that skillset. 

Jonathan