Useful Tools and Code for process watching, checking github traffic, and terraform linting

proc-watch: find out what is using the most CPU and memory

When you are running multiple EC2 instances or running a shared environment, it is useful to monitor the amount of CPU being used and how much memory is being used over time.

For example, if there's a lot of processing happening and CPU usage

• there could be something strange going on
• ...or it could be an expected amount of CPU usage depending on the workloads of the compute nodes

If there's a lot of memory usage that is unexpected

• it could be a memory leak
  ...or a new process launched that is consuming more resources
• If there are more users logging in and starting various programs, there could be more processes running or more CPU or memory usage around particular times of the days.

It can be hard to tell what's actually happening unless you SSH into the instance and check the CPU/memory usage with top or htop.

proc-watch is meant to help in this respect. It is a Python program that will parse the output of ps on Linux and MacOS systems to tell you:

• the number of processes that are currently running
• the process that is using the most memory
• the process that is using the most CPU
• the most memory being used by one process
• the most CPU being used by one process

It exports these metrics to Prometheus so that you can monitor the changes over time.

Here's how it would be useful:

• you will be able to see that the system has no load when the most CPU or memory usage is barely hovering above 0.1%
• when a resource-intensive process is started, you will be able to see that, a-ha! It's headless Chrome starting an automated test run!

Check out the code here: https://github.com/rudolfolah/proc-watch

check_tf.sh: Terraform Linting and Validation with a shell script

A small script that handles:

1. Formatting
2. Linting with tflint
3. Security check with tfsec
4. Terraform validation

You can use it as a git pre-commit hook if you setup the TFLINT_CONFIG variable.

check_tf.sh

github-traffic-check: GitHub Traffic Checker

Whenever writing blog posts or publishing video courses, I like to check in on the amount of traffic and views they are getting. It's a data point on what's interesting to readers, and sometimes suggests that I should dig deeper into a topic and write more about it.

The same is true for GitHub repos; while stars act like "bookmarks", it can be useful to see if there's anyone checking the repo and reading the source code or the documentation. It can suggest that some repos need more work to get more contributions or more usage.

Checking GitHub traffic across multiple repos was a manual process, using multiple tabs in the browser and clicking around the interface.

So, I wrote a small script using the GitHub Octokit package that prints out:

• the repository name
• the number of stars it has
• the number of views
• the number of unique views

Download github-traffic-check 1.0.0