Emmanuel Etti

Shell Cheatsheet

2023-06-01T00:00:00+00:00

Scroll through (vim) output by page

Instead of holding down up and down to scroll through a long page:

space to move down by page
b to move back up by page

Quickly move from end to start of a command prompt

Instead of holding down the left and right arrow keys to move through a command prompt

CTRL + a to go to the start of the prompt
CTRL + e to go to the end of a prompt

Search through a directory of files for a regex string content

grep -Hn -r -E 'user.+account' ./dir_path

Copy all the file contents of a directory

find /home/vagrant/dotfiles -type f -print0 | xargs -0 cp -t $HOME

Use find with flags -type f -print0 to list out all the files in the target directory with a null char delimiter instead of a new line
this allows for the selection of file names with whitespaces and such
pipe the results into xgargs with flag -0 telling it that the arguments are separated by null chars
xargs is like .map on the Unix command line, it turns what has been piped into it into a list of arguments to then call the following command on each one

References and Further Reading

Explain shell

Collection Of Helpful Guides, Talks, and Resources

2023-05-22T00:00:00+00:00

Guides

Interesting talks

Jim Weirich - Grand unified theory of software development
Any talk by Jim Weirich
- Search YouTube for Jim Weirich’s talks

Resources

Security

Ruby tools

Rails Console Cheatsheet

2023-05-20T00:00:00+00:00

See All The Methods On A Class or Instance

irb(main):001:0> User.methods

This returns an array, which one can then do array things with, like filtering based on a pattern match

irb(main):001:0>  User.methods.grep(/string_pattern/)

NOTE: The [ Enumerable#grep ] method is super useful. Check out the ruby docs to see what it can do.

Or if you want a longer way of doing it…

irb(main):001:0>  User.methods.filter {|m| m.to_s.include?("string_pattern") }

Finding A Methods Source Location

If rails constants/contexts need to be loaded, then use rails console, otherwise use irb.

# Find the source location for a method called on an instance:
irb(main):001:0> user = User.new(name: "John", email: "john@email.com")
irb(main):001:0> user.method(:send_confirmation_instructions).source_location
=> ["/Users/etti/.rbenv/versions/3.2.1/lib/ruby/gems/3.2.0/gems/devise-4.8.1/lib/devise/models/confirmable.rb", 79]

# Find the source location for an instances method 
# but through its Class:
irb(main):001:0> User.instance_method(:confirm).source_location
=> ["/Users/etti/.rbenv/versions/3.2.1/lib/ruby/gems/3.2.0/gems/devise-4.8.1/lib/devise/models/confirmable.rb", 79]

Back in the normal shell, you can then use bundler to open the gems source code and then navigate to the file and line number obtained from the previous command

# This will open up your current version of the gem
bundler open devise

NOTE: To open up a gem's source code in general can run [ gem open gem_name ] or [ bundle open gem_name ] depending on what tool is managing the dependency.

Reference: Search and debug ruby code with bundler open

Or if you want to do it an uglier waym, in the normal shell, you can take the path output, use an editor like vscode to open up the source code file and then head down to the line number given.

code /Users/etti/.rbenv/versions/3.2.1/lib/ruby/gems/3.2.0/gems/devise-4.8.1/lib/devise/models/confirmable.rb

References

Rails console deep dive

Ruby Cheatsheet

2023-05-15T00:00:00+00:00

Debugging ruby code with bundler open

bundler open GEM-NAME

This will open up the source code of the gem installed using the editor set by the environment variables EDITOR or BUNDLER_EDITOR

You can set breakpoints, change the source code, grep through it with your editor.

Once debugging is finished run bundler pristine to reset any changes

Reference: Search and debug ruby code with bundler open

Placeholder String Trick

Ruby has a trick for creating placeholders within a string and then populating the placeholders’ positions with an array of values.

VERSE = "On the %s day of Christmas my true love gave to me: %s."

VERSE % ["first", "a Partridge in a pair tree"]

The % inserts the array of values sequentially into the %s portions of the text

Thought Shaping Articles, Content, And Resources

2023-05-02T00:00:00+00:00

Software Simplicity

Software Development In The Real World

Problem Solving

Productivity

Opinions On Patterns And Best Practices

Embracing Slow

Rails Database Migration Concepts

2023-04-18T00:00:00+00:00

How Rails “Know” That A Migration Has Run

Rails Guides: ActiveRecord Migrations

How Rails “Know” That A Migration Has Run

Rails Database Migration Cheatsheet

2023-04-18T00:00:00+00:00

Dangerous Migrations (For Large Scale Production Applications)
Cheatsheet
Common Commands
Breaking Down Common Database Migration Examples

Rails Guides: ActiveRecord Migrations

Dangerous Migrations (For Large Scale Production Applications)

Certain migrations can be dangerous to perform in production because of how long they can take or how prone they are to causing exceptions if done improperly.

During a long running migration, the database tables invovled are locked, thus blocking requests to the database from the application.

The strong_migrations gem is a helpful tool for catching “dangerous” database changes. The gems’ README examples do a great job of explaining why certain migrations are “dangerous” at scale.

An alternative to using the gem is to perform any long-running/request-blocking migrations during periods of low request volume. Or to put the application in maintenance mode.

This does not avoid the other advantage of the gem, which is catching unsafe migrations that might need matching applciation layer changes.

Cheatsheet

Comprehensive rails migration cheatsheet

Common Commands

Generate a migration

bin/rails g migration AddColumnToTable

Run all pending rails migrations

bin/rails db:migrate

To migrate a specific migration out of order

rake db:migrate:up VERSION=20100905201547

Database rollbacks

Important, after creating a new migration, perform a rollback to make sure the migration is reversible

bin/rails db:rollback

Rollback a certain number of migrations

rake db:rollback STEP=5

Rollback only one specific migration (out of order) use:

rake db:migrate:down VERSION=20100905201547

Breaking Down Common Database Migration Examples

Creating a new table

class CreateUserTable < ActiveRecord::Migration[7.0]
  def change
    create_table :users do |t|
      t.string :full_name, null: false
      t.string :email, null: false
      t.string :encrypted_password, null: false
      t.boolean :admin, null: false, default: false

      t.timestamps null: false
    end

    add_index :users, :email, unique: true
    add_reference :users, :organization, 
      foreign_key: {on_delete: :cascade},
      index: true, null: false
  end
end

NOTE: Boolean columns should always have a `default: VALUE` and `null: false` constraint set

Adding a foreign key reference to an existing table

NOTE: For operations that change a pre-existing table, make sure to have the strong_migrations gem installed or check the gems README for production-appropriate alternatives to long-running request-blocking operations. These examples here are meant to help understand what can be done, not necessarily recommendations for production.

class AddForeignKeyReference < ActiveRecord::Migration[7.0]
  def change
    add_reference :users, :organization, 
              foreign_key: { on_delete: :cascade }, 
              null: false
  end
end

add_reference :users denotes the table that the foreign key column will be added to
- :organization denotes the resource the foreign key will target.
- Rails magic will pluralize this input to infer the table name organizations and add the suffix _id to infer the column name organization_id
- Scroll down to see how to add a foreign key to a table but specify the column name
foreign_key: { on_delete: cascade } tells the database to also delete child records (users) with foreign keys equal to a deleted parent record id (organization)
- If this bevaviour is not wanted, then use foreign_key: true to only add the referential db constraint that checks to make sure that the value is a valid id from the parent table
It is not necessary to specify index: true when creating a reference. Rails will add the creation SQL to the end query by default
- Creating an index for the foreign key is a performance optimization that for all the SQL queries generated by ActiveRecord when using association code like user.organization

Adding a foreign key reference with a custom column name

class AddForeignKeyReference < ActiveRecord::Migration[7.0]
  def change
    add_reference :users, :foo_bar_org, 
      foreign_key: { to_table: :organizations, on_delete: :cascade }, 
      null: false
  end
end

Alternative syntax for creating a foreign key reference

class CreateUserTable < ActiveRecord::Migration[7.0]
  def change
    create_table :users do |t|
      t.string :full_name, null: false
      
      # yada yada yada ...
      
      t.references :organization, foreign_key: {on_delete: :cascade}, 
                                  index: true, null: false

      t.timestamps null: false
    end
  end
end

Adding or removing a column from a table

Adding a column

class AddPartNumberToProducts < ActiveRecord::Migration[7.0]
  def change
    add_column :products, :part_number, :string
  end
end

Removing a column

class RemovePartNumberFromProducts < ActiveRecord::Migration[7.0]
  def change
    remove_column :products, :part_number, :string
  end
end

Understanding New Technologies

2023-04-17T00:00:00+00:00

As a software developer, it is important to continually cultivate an understanding of the tools available in the engineering toolbox. When faced with a problem, a developer with a broad landscape is better able to pick the most appropriate tool for the problem and constraints at hand.

But if one is not intentional about how they explore new technologies, they can fall into the trap of:

Chasing after industry hype
Choosing solutions based on popularity
Not taking tradeoffs into account

To prevent that, the following is a framework for learning about new (and established) technologies in a way that highlights the context and trade-offs behind them.

First, explore the questions

What problem was the technology created to solve?
What is the historical context of the technology? (what was going on around the time of the technology’s creation)
What are the alternative ways of solving the same problem?
What are the trade-offs between the other options and this technology?

Then go beyond the surface level

Read the paper, source code, docs, etc. to learn how the technology works under the hood
Do a crash course
Independently build 1-2 projects with the technology

At the end of the process, summarize the key learnings in a technical note. This prevents one from trying to memorize everything. Eventually, you may forget the details of what you learned. But when a problem arises that triggers the memory of learning the tool, you can go back to the note and asses whether the tool is a good fit for the problem.

Additional resources

You Are Not Google

SQL Cheatsheet

2023-04-17T00:00:00+00:00

Debugging Duplicates In A Dataset

Sometimes you are working on a query and you want to debug whether there are duplicates in your table.

select
  id
  count(*)
group by id
having count(*) > 1

The query will show all the rows (grouped by id) with counts greater than 1.

Unintentional duplicates can occur after a join. Instead of fixing this by using select distinct (which is a code smell), try to figure out why the duplication is happening.

Once you find the rows with duplicates, you can then select * for those records to see if there is any relationship between them that explains the duplication.

Rails Commands Cheatsheet

2023-04-16T00:00:00+00:00

General Guides
- Destroy Associated Records When Parent Records Are Deleted
Generators
- Generate The Scaffold For A Model
- Generate A Rails Model
Model Validations
- Validate email format
Misc
- A Nicer Print Out Of Rails Routes
- See all available rake tasks
Implementation Guides
Additional resources

General Guides

Destroy Associated Records When Parent Records Are Deleted

class Users < ActiveRecord::Base
   has_many :items, dependent: :destroy_all
end

Warning, there is a difference between #delete and #destroy

Generators

Generate The Scaffold For A Model

The command format is rails g scaffold <name of model> <model attributes>

bin/rails g scaffold ModelName name:string email:string description:text price:decimal

Generate A Rails Model

 bin/rails g model ModelName name:string email:string description:text price:decimal seller_id:integer img_url:string

You can undo the generation with

bin/rails destroy model ModelName name:string email:string description:text price:decimal seller_id:integer img_url:string

Model Validations

Validate email format

class User < ApplicationRecord
  validates :email, presence: true,
    uniqueness: true,

    # Uses a constant built into URI in the standard ruby library
    format: {with: URI::MailTo::EMAIL_REGEXP}
end

Misc

A Nicer Print Out Of Rails Routes

rails routes --expanded

Can pipe into grep to search for specific string patterns

rails routes --expended | grep SEARCH_PATTERN

See all available rake tasks

rake --tasks

Implementation Guides

Rails authentication from scratch

Additional resources

Lifecycle Of A Request In Rails
What Is Rack
Introducing Rack
Big idea: Rack is just 2 things
- A convention / agreed-upon protocol of how ruby “applications” should be structured so that web servers know what to expect
  - Framework developers can implement the agreed-upon Rack specs/interface that web servers would build their interface for
- A gem with a library of helper methods, modules and middleware for framework developers to use if they want
  - Instead of each framework developer building the Rack specs/interface from scratch every time, they can use Rack as a gem with all the middleware and utilities to act as the middle layer between the webservers and the framework

Protecting Data Integrity with Database Level Constraints

2023-04-16T00:00:00+00:00

The importance of database-level constraints
Common constraints
Additional resources

The importance of database-level constraints

Even though Rails comes with a way to add validations on Models it’s still a great idea to add database-level constraints as a backup to the application-level constraints (rails model validations). This helps protect database integrity when validations somehow get skipped.

Scaled applications with multiple application instances can undermine validations due to race conditions. And developers can use methods that bypass ActiveRecord callbacks (ie. validations)

Common constraints

Making sure specified attributes on records are unique

Model validation

class User < ApplicationRecord
  validates :email, uniqueness: true
end

Database constraint

When creating the table for the first time

class CreateUsers < ActiveRecord::Migration[7.0]
  def change
    create_table :users do |t|
      t.string :email, null: false
      t.string :another_attribute, null: false

      # Single attribute email to be unique throughout the table
      t.index [:email], unique: true

      # Combination of attributes to be unique throughout the table
      t.index [:email, :another_attribute], unique: true
    end
  end
end

Adding to an existing table

class AddUniqueIndexToUsers < ActiveRecord::Migration[7.0]
  def change
    add_index :users, [:email, :another_attribute], unique: true
  end
end

Making sure a column is never null

Model validation

class User < ApplicationRecord
  validates :name, presence: true
end

Database constraint

When creating the table for the first time

class CreateUsers < ActiveRecord::Migration[7.0]
  def change
    create_table :users do |t|
      t.string :name, null: false
    end
  end
end

Adding to an existing table

class AddUniqueIndexToUsers < ActiveRecord::Migration[7.0]
  def change
    change_column :users, :email, :string, null: false
  end
end

Validate based on boolean expressions

Check constraints allow you to define boolean expressions that must return truthy before a record can be saved.

In the model, you can have a validation to protect the data integrity of your attributes

class Invoice < ApplicationRecord
  validates :amount, numericality: {
        greater_than_or_equal_to: 0
  }
end

And then back this up on the database level with a check constraint

class AddAmountCheckToInvoices < ActiveRecord::Migration[7.0]
  def change
    add_check_constraint :invoices, "amount >= 0",
      name: "amount_non_negative"
  end
end

Now a database error is thrown if somehow application level validations are skipped

# Skip validations (don't try this at home)
=> invoice.amount = -1200
=> invoice.save(validate: false)

PG::CheckViolation: ERROR:  new row for relation "invoices" violates check constraint "amount_non_negative" (ActiveRecord::StatementInvalid)

Make sure associated child records are deleted when the parent record is deleted

Model-level

class Users < ActiveRecord::Base
   has_many :items, dependent: :destroy_all
end

Database-level

When creating the table for the first time

class CreateUsers < ActiveRecord::Migration[7.0]
  def change
    create_table :users do |t|
      t.string :name
      t.references :organization, foreign_key: {on_delete: :cascade}
    end
  end
end

Adding to an existing table

class AddForeignKeyToUsers < ActiveRecord::Migration[7.0]
  def change
   add_reference :users, :organization, foreign_key: {on_delete: :cascade}
  end
end

Other options for add_reference in Rails migration

null: false
index: true

Tradeoff

Since the database will delete the child records automatically without reaching out to the application, any delete callbacks placed on child record models will be ignored.

If you have specified that child records also delete certain other records when they are deleted, you have to be careful to add those constraints at the db level as well.

Chris Oliver from GoRails recommends only relying on dependent: :destroy which is slower but will make sure application-level logic is run after deletions (since the db will call back to the application).

Even the official Rails guide argues that data integrity logic should belong in your models. Seems that each use case should decide whether a model only or model + db constaints approach is best.

Additional resources

Git Concepts

2023-04-15T00:00:00+00:00

What Even Is `HEAD`?

HEAD is a symbolic reference pointing to your current place in the git history. During normal state, this points to the latest commit on the branch you are on (the tip of the branch).

When in detached HEAD state, HEAD points to a specific commit SHA.

You can see what HEAD is currently pointing to by running

cat .git/HEAD

git enters detached head state when you:

checkout a specific commit SHA
checkout a remote branch without fetching first

`git reflog` Is A Magical Time Machine

Background context

After finishing up a feature, passing review, and getting ready to ship to prod, I squashed a set of 7-ish commits and rebased onto main to clean up the branch.

Since this changes the git history, I forced pushed my changes into the remote, which forcefully replaced remote with my work.

Turns out something went wrong in my rebase and a test file for the feature I just completed (a days worth of work) had been deleted during the process. I had no copy of the commit introducing the test file in my local and no copy in the remote.

Turns out git reflog was able to help me find the SHA of the commit where the test file was. I checked out that commit SHA and copied the test file into my squashed feature branch. I probably could have used git cherrypick but that is a lesson for another day.

What is git reflog

The reflog is a log of all the moves HEAD has made in your local repository. Every time you switch branches, start a rebase, merge branches, go into detached head state, etc. the reflog keeps an entry of this movement. Each entry includes the commit hash, and a message that contains a few other infos (like the branch name you switched to). This data is stored in your projects local .git directory.

The reason why it’s possible to restore a messed-up repository with reflog is because git keeps those commit hashes. As long as there is some reference to a commit somewhere in the repo, Git won’t forget about that commit.

As long as you are following good commit practises - commiting often, committing atomically - it’s near impossible to have something be lost forever.

Just don’t delete your .git directory, the reflof of all your local movements will NOT be in the remote repo.

Additional Resources

Git Cheatsheet

2023-04-15T00:00:00+00:00

Clean up local branches
Reset a commit from HEAD
Edit the changes within a commit without destroying the commit
Un-staging a file
Show a condensed log of commits
Show a log of commits with cool graphs
Compare the changes between commits
Further readings

Clean up local branches

You’ve been working hard and now you have a bunch of local feature branches that you don’t need anymore. Instead of running git branch -D on each one.

git checkout main

git fetch

git branch --merged | grep -vE '(\*|main|master)' | xargs git branch -D

git checkout main switches the current branch to “main”. This assumes that the “main” branch already exists in the repository.
git fetch downloads the latest changes from the remote repository to your local repository. This ensures that you have an up-to-date list of branches that have been merged into the “main” branch.
git branch --merged lists all the branches that have been merged into the current branch.
grep -vE '(\*|main|master)' filters out the current branch (*), as well as any branches named “main” or “master”. The explicit filtering of main and master is paranoid insurance … just in case the initial checkout didn’t work for some reason.
xargs git branch -D passes each of the remaining branch names to the command git branch -D, which deletes them one by one. xargs is like the map method in a Unix command line.

Reset a commit from HEAD

You just committed some changes but want to undo that commit for whatever reason. Maybe you’ve committed to the wrong branch, or you forgot to add another change, etc.

git reset HEAD~1 --soft

This will destroy that git commit! But with the --soft flag the changes in that commit will be preserved for you to create a new commit (or set of commits) from.

If you want to reset the last N commits (from the current HEAD) while saving the changes from those 5 commits. Replace N with whatever number you want to go back.

git reset HEAD~N --soft

If you want to reset changes on the branch back to a specific commit from HEAD (while still keeping the changes)

git reset --soft <commit hash>

⚠️ Warning

This action re-writes the git history and should be done with care. As a rule, do not make changes to the git history on work that has already gone public. This breaks things for other devs working on that branch. Only do this if:

you haven’t pushed the changes to the remote repository
you know you are the only one working on that branch

Edit the changes within a commit without destroying the commit

You want to add some new content or edit some of the files changed in a specific commit.

git rebase -i main OR SHA_OF_SPECIFIC_COMMIT_TO_REBASH_OFF_OF

This will open up an interactive rebase session where you will see all the commits your branch adds on top of the tip of main or a specified commit SHA.

For each commit, you will have a set of actions you can take. Choosing edit will place you into that commit where you change the state of the files within that commit. You can create new files, change modifications, delete files, etc.

Be careful with the changes, as you may create git conflicts with any following commits.

Once changes have been made:

verify the changes made with git status and git diff, or using the vscode source control UI run git add . OR specify-the-files-to-add to add the changes, and then continue the rebase
run git add . OR specify-the-files-to-add (be careful with git add .)
run git commit --amend
- since we did not destroy the commit with git reset HEAD~, the commit still exists but now we want to amend all these new changes to that commit
- If we did reset the commit (destroy it) DO NOT RUN git commit --amend as this will amend all your changes to the last previous commit which is NOT the old commit anymore since we blew that up with the reset
But if you want all the new changes in their own atomic commit then do the normal flow of
- git add . (use carefully)
- git commit

Once complete, run git rebase --continue to continue the rebase.

Remove the changes made to a file

git status # to get the path names of the files added
git reset -- FILE_PATH_NAME # to remove the file

rm FILE_PATH_NAME # if you need to delete the file all together

Un-staging a file

You yolo’d and used git add . without checking the changes included. But thankfully you have not committed yet.

git status # get the name and path of the file
git reset -- FILE_PATH # unstage the file from staging

Show a condensed log of commits

git log --oneline

Show a log of commits with cool graphs

git log --oneline --decorate --color --graph

Compare the changes between commits

Compare the current commit and n number of commits back

git diff HEAD~n HEAD

Compare the current commit with the tip of main

git diff HEAD main

Personal Software Developer Growth Plan

2023-04-07T00:00:00+00:00

Introduction
Fundamentals of a full stack web developer
Fundamentals of software craftsmanship
Fundamentals of computer science
- Resources
- Low-level programming
Healthy reminders and inspiration

Introduction

Knowing every framework, tool, language and library is not possible, nor is it the goal of a software engineer. This is a continually improving personal plan to help me grow as an engineer capable of solving meaningful problems.

Resources

Problem solving practise

Learning platforms

Practice

Build, build, build, then get feedback
Build independent projects
Read the source code
Peek under the hood

Fundamentals of a full stack web developer

The goal is to be able to create and deploy an end to end web application from scratch; as well as contribute effectively to an established project.

How the web works

Intro

Deeper

Full stack web development

Intro

Deeper

Web relevant programming languages

Ruby

Javascript

Battle-tested web framework (rails)

Intro

Deeper

Marc-Andre Courmoyer - Owning Rails
Refactoring Rails
Noah Gibbs - Rebuilding Rails
Destroy All Software - Web Framework From Scratch
Paul Dix - Service-Oriented Design with Ruby and Rails
Arkency - Domain Driven Rails

Front-end library (react)

Intro

Deeper

Rebuilding React
Ilya Grigorik - High Performance Browser Networking
Michailia Verou - CSS Secrets

Databases and datastructures

Intro

Deeper

Botros & Tinley - High Performance MySQL
Build a DB from scratch

Core tools

Git

Terminal

Text Editor

Browser dev tools

Freecodecamp - Chrome DevTools

Developer experience tools

Offline documentation browser:
- Devdocs
- Dash for macOS
API development: Bruno
Database GUI: Table plus
General productivity: Raycast

According to the Pragmatic Programmer book, aim to do the following actions using only the keyboard with your text editor of choice

Edit text by selecting by character, word, line, paragraph

Move selected character, word, line, and paragraph

Find exact/partial matched word, pattern, delimiters, function, etc

Indent and reindent block of code

Comment and uncomment blocks of code

Undo and redo changes

Split the editor window into multiple rows or columns, and jump between them

Go to a specific line number

Sort selected lines by descending and ascending

Create multiple cursors to select and edit the text in parallel.

Open new terminal and problem windows in the current project.

Start service and run all/specific test cases in project

Fundamentals of software craftsmanship

General craft

Object Oriented Programming

Sandi Metz - Practical Object Oriented Design In Ruby
Avdi Grimm - Mastering The Object-Oriented Mindset In Ruby And Rails
Rebecca Wirfs-Brock, Wilkerson, Wiener - Designing Object-Oriented Software (also see this PDF)
Matt Weisfeld, The Object-Oriented Thought Process
Freeman and Pryce - Growing Object-Oriented Software, Guided by Tests (GOOS) #goos-bookclub
Eric Evans - Domain Driven Design
Vaughn Vernon - Implementing Domain Driven Design
David West - Object Thinking

Design and software architecture

Robert Martin - Clean Code
Robert Martin - Clean Architecture
Russ Olsen - Design Patterns In Ruby
Gang of Four - Design Patterns
Freeman, Baters, Sierra - Head First Design Patterns
Martin Fowler - Domain Driven Design
Martin Kleppmann - Designing Data-Intensive Applications
Roberto Vitillo - Understanding Distributed Systems
Ford, Richards, Sadalage, Dehghani - Software Architecture: The Hard Parts
Robert C. Martin - Clean Architecture

Refactoring

Martin Fowler - Refactoring
Jay Fields - Refactoring Ruby
Joshua Kerievsky - Refactoring To Patterns

Testing

Thoutbot: Testing Rails
GoRails: Testing Series
Micheal Feathers - Working Effectively With Existing Code
Gerard Meszaros - XUnit Test Patterns: Refactoring Test Code

Security

Intro

Deeper

Lewandowski, Adkins, Beyer, et al - Building Secure and Reliable Systems
Gilam & Barth - Zero Trust Networks
Cliff Stoll - The Cuckoo’s Egg: Tracking a Spy Through the Maze of Computer Espionage
Michael Zalewski - Tangled Web: A Guide to Securing Modern Web Applications
Yaworski - Real-World Bug Hunting
Stuttard & Pinto - The Web Application Hacker’s Handbook
Sandworm: A New Era of Cyberwar and the Hunt for the Kremlin’s Most Dangerous Hackers

Performance

DevOps

Site Reliability Engineering: How Google Runs Production Systems
Murphy, Rensin, Kawahara, Throne, Beyer - The Site Reliability Workbook: Practical Ways to Implement SRE
Brendan Gregg - Systems Performance: Enterprise and the Cloud
Stevens & Rago - Advanced Programming in Unix Environment
Kim, Debois, Willis, Humber - The DevOps Handbook
Michael Nygard - Release It!: Design and Deploy Production-Ready Software

Fundamentals of computer science

This is the lifelong journey of learning the computer science foundations that will stand the test of time.

Resources

Low-level programming

Recommendations from co-workers on low-level programming resources.

Healthy reminders and inspiration

Peak under the magic

Start to peek underneath the “magic”
Re-build the tools, read the source code instead of the docs, and contribute to the open source project
Look for courses that teach how to build the tools from scratch in addition to courses that teach how to use them
Play around with the source code and see what breaks

There is no rush

It’s a marathon, not a sprint
Software development is a craft, not an academic study
The only way to get better is to learn by doing, building, and getting feedback on your work
No one can learn everything there is to know in software
Don’t try to memorize things
It’s okay to take weeks to months-long breaks from learning if it’s becoming too much or the season of life is too full
Make sure to protect the fun

The place of books

Books are collections of learnings that authors have gained over time by actually doing things and making the mistakes
Books are not a primary way of learning craft-based skill sets
Books are there to refine a craft using knowledge, mistakes, and learnings from those that have gone before
You do not learn to ride a bike by reading about it
Improve the consumption of books by stopping just before the author implements the solution and first really trying to implement it yourself. Then compare to see how the author did it

The place of courses

Courses are also not the primary way of learning craft-based skill sets
Courses are there to get you started or introduce a new concept or tool
Can improve the consumption of course by building along with the instructor
The goal is that after the course you then build independently
Improve the consumption of courses by stopping just before the instructor implements the solution and first really trying to implement it yourself. Then compare to see how the author did it

Defining A Good Software Developer

2023-04-06T00:00:00+00:00

Introduction
Software development as house building
Having a product mindset
Solving problems, not writing code
Making good software
Working with other humans
Pitfalls

Introduction

“You did not learn to ride a bike by reading about it. You learnt by jumping on a bike and falling a lot. Over time you figured out how to stay upright and steer. In the same way, you can only truely learn to become a better programmer by writing a lot of code. In the beginining that code will be bad. You will make a lot of mistakes. But then you will learn from all those efforts. Books are just a collection of the hard lessons authors have learnt over thier careers as working engineers.”

– Senior Engineer @ Shopify

Software development as house building

A house is meant to solve a real problem for someone. It provides shelter from the elements, storage for belongings and supplies, and grows with the changing needs of the occupants.

Houses are more than a combination of a roof and four walls. There are other concerns such as electricity, plumbing, heating, insulation, aesthetics, and more. Every decision made when building the house, from the design to the materials chosen, comes with tradeoffs.

Any contractor worth their invoice can build a house. What separates a good contractor from a bad one, is the considerations and care taken in the craft of house building.

Imagine two beautiful homes built by two different contractors. House A is built by Jane, an experienced contractor who’s learned some important lessons throughout her career. House B is built by Joe, a contractor who does not yet have the same level of experience as Jane.

At first glance, both houses look amazing. But when the occupants of Joe’s house move in, they find that some details have been missed and some cupboards are misaligned - okay not so bad they think.

After a few weeks, they find that the plumbing stops working at random times of the day. Kind of annoying. After a few months, they decide to install a new hot water heater. But the schematics that Joe left them are difficult to make sense of. The owner decides to ignore the schematics and look at the pipes directly to trace how they’ve been put together. But the implementation is such a black box that Joe needs to be brought back to do the installation himself.

After a few years, the family of Joe’s house have had new family members move in - their needs have changed. They would like to renovate part of the house to accommodate their new situation. Having learned their lesson, they decide to hire Jane to oversee the renovations and expansion. She tells them that, unfortunately, the structural layout of the house and the placement of key electrical and heating components are so fixed that making the changes will require an entire rebuild of the house.

Even though both Jane and Joe accomplished the fundamental goal of a contractor (building a house that meets the needs of the occupants), Joe’s house was not:

Flexible (easy to change)
Reliable
Maintainable
Understandable (easy to reason about)
Scalable with growing the needs of the occupants

Having a product mindset

The first and most important goal of a good software developer is to remember that the business value of their role is to provide solutions to business problems. This is where both Jane and Joe accomplished their main goals.

Does their work contribute to:

Reducing the time and financial cost of operations
Increasing product quality
Increasing the customer base
Increasing the number of products to sell
Innovating and creating new product categories to sell

This is in opposition to seeing software development value solely as:

Implementing the most cutting-edge technology
Constantly shipping product features (many features do not make a good product)
Accumulating technical expertise (this will naturally happen over time)

Solving problems, not writing code

Although it may sound surprising, the primary focus of a software developers job is not writing code but rather creating value through the use of software that was written. Code is simply a tool to achieve this end goal. Code -> Software -> Value.

What you write needs to fill some need in the world — some tool that users will use, some automation that reduces costs, something people will pay for (with their time, money, or attention). We can simplify it. If you build something with shitty technologies that provides great value to the users — you’ve served your purpose as a software developer. If you’ve built something with great technology that offers shitty value to the user — you didn’t.

Elegant code, best practices, smart solutions, design patterns — these are done for the sake of your fellow software engineers who will work on the codebase after you rather than helping you fulfill the purpose of bringing value. (Mind you, bringing value can also mean building a scalable solution that doesn’t crash, which requires the code to be at least somewhat decent.)

– Vamid Kravcenko

Making good software

Making good software is an art versus a science. It’s learned over time and with experience. Many of the lessons learned will be through painful and expensive mistakes, and that is okay.

Security

One of the fastest ways to lose customer confidence is through a security breach. People trust companies with their private data and engineers owe it to them to do everything possible to learn and optimize for security. Rely on the simple and battle-tested solutions over the fancy and brittle.

Flexibility (easy to change)

The only constant in software is that things will change. Business requirements will change, the team with learn more about the domain, new features will need to be added, etc.

All the considerations of patterns and best practices are about making software easy to change. Software systems that are tightly coupled, not well tested, complex, and hard to understand are painful to change. The goal is not to optimize for the computer but to optimize for the humans who will make changes to the software.

Simplicity

Build simple solutions. Avoid clever tricks, “elegant” one-liners, or using the newest trendy tools. Simple solutions, with no more moving pieces that are needed, are much easier to maintain by a wider variety of people and skill sets.

Sometimes software developers can even achieve the primary goal of finding solutions to business problems without writing any code at all.

Understandability (easy to reason about)

Don’t build black boxes.

When an event happens in the system, team members should be able to logically piece together what happened and communicate exactly why it happened. Anyone looking at the architecture diagram or logic flow should be able to make sense of how it’s put together.

Black boxes can look like convoluted methods and functions, confusing components, poor documentation, and overly complex system architectures. Just like the simplicity principle, strive to make solutions easy to hold in your brain.

Understandability can be improved with a mindset of building for the next person to maintain. When building, imagine yourself having to dive into the software a year later to fix a time-sensitive bug.

Reliability

Reliability is all about how often your system breaks, how easily your system can break, how your system handles itself when things go wrong, minimizing data loss and business interruptions when systems break, and the practices around getting the system back online and building back better.

Maintainability

“If something blows up, we need to have made it possible and easy for the correct state of the system to be recoverable, either by the system itself or by those operating the system”

– Senior Engineer @ Shopify

Good software should strive to be easy to maintain. Writing easy-to-read code, building simple solutions, avoiding “clever tricks”, keeping updated documentation, good git commit practices, and writing good tests should help with making a system easy to maintain.

Scalability

Scalable software grows and shrinks to meet the demands of the business. But be careful with premature and untargeted optimizations. These are the root of all evil.

Working with other humans

Attitude

Be kind. Be humble. Be curious. Ask LOTS of questions. Fight against FOLD (Fear Of Looking Dumb). Treat others as you would want to be treated.

Communication

⚠️ Work in progress

Productivity

Focus on one needle-moving task per day.

Additional resources

Pitfalls

Trying to memorize everything, learn everything, then burning out

The value of a software developer is not measured by how much technical knowledge a person has accumulated. Technical knowledge comes, goes, and accumulates as a person navigates a career and the different projects they are exposed to.

Software developers are professional problem solvers that use computational tools and engineering craft to do needle-moving amounts of “work” for other people. This is done via a systematic way of identifying, breaking down, and clarifying problems to their smallest components. Then researching potential solutions to the problems, while exploring trade-offs. And then (the final 20% of the time) implementing the solutions.

Pick a core set of tools, and master them. When researching other solutions, it’s important to timebox the process to keep from rabbit trailing and going deeper into the technology than the problem warrants.

Instead of trying to learn everything at once, focus on knowing enough to solve the problems in front of you. Trust that when you need to solve a problem, you will be well-equipped to learn what’s necessary to solve it (Just In Time learning).

Follow your curiosity and spend extra time outside of work projects learning new things. Focus on understanding the “shape” of the thing. Some questions to ask while exploring a new technology or topic:

What is it?
How does it work under the hood?
What is the historical context around this thing?
What problem is it meant to solve?
What are the alternative ways of solving the same problem?
What are its trade-offs?
Can I build a prototype with this thing?

Chasing the shiny, new, and popular

Boring is good. A lot of the new and shiny technologies in the industry have been built to solve extremely niche problems for large tech companies. for 80% of use cases, the old battle-tested solutions are perfectly fine.

Many of today’s $10M+ businesses are sitting on “boring” tech stacks:

Ruby (est. 1996)
Monolithic Rails (est. 2004)
Postgresql (est. 1986)
Redis (est. 2009)
Sidekiq (est. 2012)
Heroku (est. 2009)

Instead of chasing tutorials for the newest framework, tool, or technology, focus on mastering the foundational things that have stood the test of time.

But don’t completely ignore the new and shiny. As with most of life, moderation is key and it’s nice to explore out of curiosity.

Not thinking in terms of tradeoffs

Every decision has tradeoffs - nothing comes for free. A good practice is writing RFCs for major decisions to help process the pros and cons of a decision.

Blindly following “best practice” (without understanding the why)

Understand the “why” behind the best practice. This will help with gauging when it’s appropriate to break the best practice. For example:

Blindly following DRY may result in picking the wrong abstraction too early
Blindly following YAGNI may keep you from building your software in a way that anticipates coming changes

Rushing to ship

Rushing to ship can mean putting out software that does not meet the above considerations of “good quality”.

Whatever time and money is saved in today’s shortcuts will be small in comparison to the time and money lost in tomorrow’s bugs, customer frustrations, and painful feature additions.

The Value of a Software Developer

2023-04-05T00:00:00+00:00

Introduction
The computer
The internet
The general importance of these tools
The business importance of these tools
Conclusion

Introduction

“So the reason programmers sometimes get paid a lot of money, I think, is because we can build systems that leverage computers to do an unreasonable amount of work. If you build Gmail’s spam system, you can remove spam from the inboxes of millions of people! This is kind of magical and amazing and it’s worth all of the bugs and dealing with computers.”

– Julia Evans

An important part of being human is the drive to create. Beautiful things, destructive things, physical goods, abstract ideas.

The computer

One incredible thing we have made is the computer. A computer is a machine that performs sequences of calculations and logical operations many times faster than the human brain. Operations can be combined into collections of logic meant to accomplish specific tasks. These are called software programs.

Over the years, we have built up layers of abstractions to create the modern computing machines that we use daily. Instead of teletyping binary instructions into room-sized mainframe machines to perform scientific research calculations; we click, drag, draw, and type into portable laptops made up of hardware components and software systems to accomplish general-purpose productivity tasks.

The internet

The next major invention came when we linked these computing machines together to form a network called the Internet. This allowed computers to pass information between each other across physical fiber optic cables.

Today a person can write a set of instructions (software) on their Macbook Pro in a trendy co-working space in Vancouver, BC, and deploy those instructions through the internet to hundreds of specialized computers in a Google data center in New York, where they are executed and the results made available to the rest of the world.

The general importance of these tools

The reason computers, connected via the internet, have become so important to society is that they make it easier for us to fulfill that core human drive to create.

Computers can help us build tools that:

Make life easier for individuals
Help businesses operate more efficiently. A network of computers can h
Allow us to share information and disperse ideas around the world

The combination of human creativity, curiosity, and millions of transistors, has meant that creating things requires a lot less resources than it used to.

The business importance of these tools

A business is nothing more than an individual or team solving a problem for a target customer base. The problems being solved are important enough to the customer base that they are willing to give money (a societal representation of value) in exchange for the solutions. A family will pay a monthly subscription to Netflix because Netflix’s product is solving their problem of boredom.

If some of these solutions can be turned into systems of logical operations and calculations (software programs), then they can be performed by computers - which need no sleep, food, or water. Once created, these software systems can serve a small niche of hundreds of people or be scaled to meet the needs of millions of people around the world.

Even if a business’s primary way of providing solutions to others can’t be turned into software, many functions within that business can be made a lot easier with the help of computers. Think of the payroll software that powers your local garbage disposal company.

The reason automating a simple payroll process is valuable is because of its effect on one or more of the main concerns every business has:

Reducing the time and financial costs of its operations
Increasing the quality of its products
Increasing the number of products it creates
Innovating and creating new product categories to sell
Increasing the number of customers it serves

Conclusion

The individuals who can build and maintain these software systems are the software developers. They work with entrepreneurs (people dedicated to finding profitable problems to solve) or decision makers within established organizations to take “business” problems, clarify them, and present solutions made up of software systems that are:

Flexible (easy to change)
Reliable
Maintainable
Understandable (easy to reason about)
Scalable

Emmanuel Etti

Shell Cheatsheet

Scroll through (vim) output by page

Quickly move from end to start of a command prompt

Search through a directory of files for a regex string content

Copy all the file contents of a directory

References and Further Reading

Collection Of Helpful Guides, Talks, and Resources

Guides

Interesting talks

Resources

Security

Ruby tools

Rails Console Cheatsheet

See All The Methods On A Class or Instance

Finding A Methods Source Location

References

Ruby Cheatsheet

Debugging ruby code with bundler open

Placeholder String Trick

Thought Shaping Articles, Content, And Resources

Software Simplicity

Software Development In The Real World

Problem Solving

Productivity

Opinions On Patterns And Best Practices

Embracing Slow

Rails Database Migration Concepts

How Rails “Know” That A Migration Has Run

Rails Database Migration Cheatsheet

Dangerous Migrations (For Large Scale Production Applications)

Cheatsheet

Common Commands

Generate a migration

Run all pending rails migrations

Database rollbacks

Breaking Down Common Database Migration Examples

Creating a new table

Adding a foreign key reference to an existing table

Adding a foreign key reference with a custom column name

Alternative syntax for creating a foreign key reference

Adding or removing a column from a table

Understanding New Technologies

First, explore the questions

Then go beyond the surface level

Additional resources

SQL Cheatsheet

Debugging Duplicates In A Dataset

Rails Commands Cheatsheet

General Guides

Destroy Associated Records When Parent Records Are Deleted

Generators

Generate The Scaffold For A Model

Generate A Rails Model

Model Validations

Validate email format

Misc

A Nicer Print Out Of Rails Routes

See all available rake tasks

Implementation Guides

Additional resources

Protecting Data Integrity with Database Level Constraints

The importance of database-level constraints

Common constraints

Making sure specified attributes on records are unique

Making sure a column is never null

Validate based on boolean expressions

Make sure associated child records are deleted when the parent record is deleted

Additional resources

Git Concepts

What Even Is HEAD?

git reflog Is A Magical Time Machine

Additional Resources

Git Cheatsheet

Clean up local branches

Reset a commit from HEAD

Edit the changes within a commit without destroying the commit

Un-staging a file

Show a condensed log of commits

Show a log of commits with cool graphs

What Even Is `HEAD`?

`git reflog` Is A Magical Time Machine