Systems & Programming Fundamentals

A roadmap to get a start in the world of Systems and Computer Programming

Roadmap Overview

Preface

This roadmap is a personal project that I'm working on as a way to document my journey learning about the world of systems and programming. While I have done some self-studying and know a few things about systems, I'm nothing more than a jack of all trades with shallow knowledge. But all of the knowledge and experience I've gained over the past 3 years has led me to creating this roadmap that I plan to follow.
When I first started to self-study programming, I knew nothing about what was out there, I knew that I was interested in software engineering and robotics, but I hadn't the slightest idea of how to get there. Now I know the path I need to follow to get there, but first I wanted to pursue my interests in Systems, Networking and General Programming.
That's when I realized that this was the perfect time to master the fundamentals starting all the way from System Administration with Linux to covering Computer Networking then Cloud Technologies and finally diving right back into Computer Science.

I am sharing this roadmap of mine as a way to showcase my learning journey starting from Systems & Networking to Computer Engineering and I hope that others who read this may find something that they can take away for themselves.

The Main Topics

The roadmap, as of now, consists of 5 main topics with each major category containing their own sub-topic.

The intended flow of the roadmap is to go through each major category and work on every sub-topic then do the associated projects for each section. All of this culminates in a final capstone project that will put your understanding of the category to the absolute test.

roadmap.sh explanation

Roadmap_Explanation.png

The philosophy of this roadmap is taking the long-game approach to studying. A big mistake we often make in learning is that we will cover a subject just enough to scrape by. While you don't have to know everything about a subject to be able to do interesting things, when it comes to our foundations true understanding is essential.

So, to remedy this, we will cover each major topic in just enough depth to get a strong grasp of the basics. The blue button behind every sub-topic will link to the full article in the Lab-Book. Meanwhile the actual sub-topic will cover a brief summary of the course, video, or book we will be working through. It will also include relevant resources. While not pictured here, the Alternatives section will provide other courses/videos that work as a roughly equal substitute, but they may also both be taken.

The best way to truly cement one's knowledge on a topic is by doing, and so the turquoise-colored checkmark ✅ next to the sub-topic will be where we begin our section project work. Lastly, we arrive at the Capstone Project ☑️, this is a section dedicated to putting all of the knowledge we have learned to the test. The Capstone Project will either be a single large project or a series of projects that puts everything we have learned together. Lastly while not pictured, there will be a dark-blue checkmark next to courses/books that are optional but may be worthwhile depending on your use-case.

✨Personal Note

Throughout this journey I will do my best to be as objective as I can. However, I am still in the middle of my learning and I am only human. While technology itself is neutral, the way we use it, how we think about it, what we think is best moving forward and so on, are things that are human viewpoints. There are of course times when things are simply 0 or 1, but as you go through your own journey you will find that you will have to come up with your own definitions and reasons for doing things backed by personal preference or well researched analysis or simply that's the way things are done. 
So, to carry on with that spirit I will place the sparkling star icon✨ whenever certain notes or topics are things I am speaking my own mind on. I encourage you to come up with your own analysis or thoughts on those topics.

The Roadmap

In The Roadmap below. All of the time estimates are based on studying the material for 2-3 hours per day.
Legend
- 🔄️: Alternatives | - ⏭️Optional 

System & Networking Fundamentals

A system can be defined as a set of parts working together, and a network as a group or system of interconnected things.

The aim of this path is to familiarize ourselves with Linux & Computer networking. We'll learn to both configure and administer our very own Linux Servers, write powerful scripts, and come away with a sys-admin’s eye and toolset. We’ll also gain a solid foundation in computer networking, uncovering the Blackbox that the internet often seems to be, learning about various protocols, topologies, and architectures.
At the end of this path, we’ll be able to call ourselves a jr sysadmin and jr network-engineer.

Linux Fundamentals

Networking Fundamentals

Intro to Programming & Automation

This is by no means an introduction to computer science, that will be something to cover later. Instead, this path is all about easing us into computer programming. We'll be learning about the absolute basics of programming and some automation as well. We'll also cover Databases learning about CRUD and Data-In/Data-Out, then finally rounding things out with version control.
By the end of this path we'll have taken our first steps in programming.
There's still a long way to go before we can call ourselves software developers much less engineers, but bit-by-bit we are getting there.

Intro To Programming

Containers & Orchestration

Docker

Kubernetes

Intro to Computer Science

Programming

Hardware Fundamentals

Computer Systems

Systems

Linux Programming

Linux Fundamentals

Linux Fundamentals

The Linux Upskills Challenge

LinuxUpskillChallengeTitle.png

Course Overview

We begin our journey into Systems and Programming with the Linux Upskill Challenge  ( GitHub )  

This challenge is a great way to take our first steps into Linux. It is a month long commitment that you can follow at your own pace, but if you dedicate 2-3 each day it will take you roughly 21 days to complete the challenge.
For our purposes we will be using the Challenge to as a way to learn Linux for the first time and to also begin building on our Linux Command Line Skills.

Goals of this Course
Course Notes

The Linux Upskill Challenge has a lot of great concise and detailed explanations for all of the topics they cover. Which makes it pretty hard to make any notes that wouldn't just be a copy & paste.
As a middle ground the following notes are commands and information I found particularly useful.

A quick note for people who are windows users!

Some of you may be using WSL to access your cloud instance. You might experience clock-drift on your device making you unable to access to the cloud CLI. If this happens try running the following command in the block below. 
This should serve as a temporary fix to your clock-drift but I would recommend looking online for a more permanent solution.

sudo hwclock -s

The ls command and its switches
    - ( -l ) : lists directory contents
    - ( -a ) : lists hidden files
    - ( -t ) : lists by date created
    - ( -r ) : lists in reverse order
Some notes on Sudo & Root Best practices 
    - Don't leave root open
    - Be careful of using sudo to open GUI applications, if you have to, use sudo -h 
    - /etc/shadow : A very important space that stores information about the user passwords on a Linux system
    - Hackers can't auto-crack if root is locked down
Commands used to find files
    - locate | find | grep | which
Ownership and Permissions in Linux
    - ls -ltr is a good way of showing who owns what files and what permissions they have
    - chmod
         - u = user | g = group | o = others | a = all
         - ( + ) = add permission | ( - ) = remove permission | ( = ) = set permission
         - r = read | w = write | x = execute
         - u = user | g = group | o = others | a = all
         - 1 = execute | 2 = write | 4 = read
examples:
         - chmod u+w hello.txt | Gives user write permissions to the hello.txt file
         - chmod 777 hello.txt | Gives read write and execute permissions to everyone
Important File Transfer protocols
Linux can share files through a variety of protocols. These 3 are key protocols to keep in mind
         - SMB: Microsoft file sharing, useful on a local network of Windows Machines
         - AFP: Apples file sharing, useful on a local network of Apple Machines
         - SFTP: Access & transfer files over SSH
You can get practice with vim by running vimtutor
Turn a file into a bash script with #!/bin/bash

Required Reading

The required reading section in the Linux Upskill Challenge covers all the material you need to get started. I added setting up SSH login as required since the challenge lists it as an extension.

Similar to before, the recommended readings posted on the challenge are very good and definitely worth going through. The 2 articles below are extensions that I found particularly interesting or practical.

Section Projects

Project 1 - Take your first steps into the world of Self-Hosting

What is Self-hosting? 
Self-hosting is the practice of running and maintaining a website or service using a private web server, instead of using a service outside of someone's own control.

We should now be knowledgeable enough about Linux to comfortably follow tutorials on self hosting.
The world of self hosting is a rabbit hole and there are more services out there than you will ever have time to understand and use, which also means that there's bound to be something for everyone.

For this project your task will be to self host 3 services

The first 4 services I listed are the ones I first set up when I finished the challenge. WordPress and Pi-hole were relatively straightforward but getting Grafana & Prometheus to work was definitely a challenge.
I also linked to the Awesome Selfhosted GitHub page that shows 100's of services you can self host.

Linux Fundamentals

Learning the Unix OS

LearningTheUnixOS_5th_Edition.jpgLearning the Unix OS - 5th Edition
Jerry Peek, Grace Todino & John Strang ( O'Reilly, 2002 )

This book is now over 20 years old, yet it has aged gracefully.
Roughly 90% of the information inside is still relevant today and the other 10% can be treated as a history lesson. 
The reason can be found in the book's preface.

This book teaches basic system utility commands to get you started with Unix... We cover a commands most useful features...

And it more than lives up to its purpose. While Linux & Unix have changed a lot over the years the command line itself hasn't changed radically. The reason why the commands shown are interchangeable is that many of the core commands are identical in terms of syntax and functionality between Unix & Linux. There are of course some differences, but they are not enough to be a problem at our current level. We won't learn about Modern Filesystems or Containers, but we will learn enough to get even more confident with working through Linux.

Book Overview

Our next step in the systems fundamentals course will be Learning The Unix OS by O'Reilly. You may treat this book as a Practical Guide to Unix. This book is 157 pages long and will take you 7 days to finish if you dedicate between 2-3 hours every day to work through it.

When you look at the title you will notice the word Unix, but don't let the word Unix discourage you. Much of what we will be going over will be regarding the Command Line Interface. Which for our purposes is practically interchangeable with Linux. Don't misunderstand, they are two different operating systems, with the key difference being that Linux is Open-Source while Unix is Proprietary

The main goal of this book is to serve as a Practical Starter Guide, covering the main essentials of the CLI and Unix OS. That is why this book is still remains relevant even today. While we did go through an introductory course already, that being the Linux Upskill Challenge, there are some nuances that that this book goes through that will be very helpful to us.
We'll explore the various types of commands, Windows Systems, basic File Management, I/O-Redirection and even some interesting pieces of Linux history. When I finished going through this book, I felt confident that I could start working on this roadmap and bring something interesting to the table. Although we still have a long way to go before we can call ourselves true adepts, but steadily, we are making progress.

✨Personal Note

( The following has been added to the overview page. But I am keeping this here as this was when I thought of the idea. )
Throughout this journey I will do my best to be as objective as I can. However, I am still in the middle of my learning and I am only human. While technology itself is neutral, the way we use it, how we think about it, what we think is best moving forward and so on, are things that are human viewpoints. There are of course times when things are simply 0 or 1, but as you go through your own journey you will find that you will have to come up with your own definitions and reasons for doing things backed by personal preference or well researched analysis or simply that's the way things are done. 
So, to carry on with that spirit I will place the sparkling star icon✨ whenever certain notes or topics are things I am speaking my own mind on. I encourage you to come up with your own analysis or thoughts on those topics.

Goals of this Book
Course Notes

I quite enjoyed making the notes for this book since it led me to making several web searches on various different topics about Linux both old and new. 

Required Reading

Our required reading for this section will be a mix of reviewing old commands with learning new ones. We begin by covering some particularly useful List and System/Network monitoring commands. For those I recommend using the man page in order to get more comfortable with using the man command. After that we'll cover some articles on the Digital Ocean and RedHat community blogs that cover some more useful commands.

✨Finally we end with a more philosophical article, technical ability is important but I encourage you to consider the abstract side as well. Primarily questions like: "Why is Unix & Linux designed this way?" "What the point of FOSS?" and musings of that nature. 

( Self-Guided ) - Look into the following commands and go through them by using the man command. Read the Name, Synopsis, Description and go over some of the Options, then try the commands out. For system memory commands I recommend paying attention to the -h flag which shows displays the output in a human readable format. After you read the command's man page if it's use-case or meaning is unknown to you, do a web-search for more information.

( Article ) Digital Ocean Community: Top 50+ Linux Commands by e...f - By this point we have gone over most of the commands listed in the Digital Ocean article. I recommend reviewing any commands you're not confident in and pick up any that you may not know.

( Article ) RedHat - Enable Sysadmin: Linux Commands: du... by Tyler Carrigan - The du command is an excellent tool that will give you insight into how your storage is being used. This short article will go over the most used flags and how to make the most of them. 

( Article ) RedHat - Enable Sysadmin: How to Manage Linux Permissions... by Damon Garn - Managing user permissions is an essential task for any sysadmin. User access control is an essential security feature meant to protect data by giving only certain users access to data and resources based on certain contexts. If you still feel unconfident about managing permissions focus on this post and make sure to understand absolute/octal mode.

( Article ) Linux-Databook: The Unix and Linux Philosophy by dboth - Every Operating System has it's own philosophy. To steal the quote listed in the post. 

An operating system, by its nature, embodies the philosophy of its creators…"  - Mike Gancarz

Essentially any OS from Linux to Windows to macOS has their own way of doing things. It's Linux's philosophy that makes it so powerful in a capable users hand. As always I recommend you take away your own conclusions from this article. You may agree with much of it now and maybe come to disagree with much of it later and vice versa, even I don't agree with every tenet.

( Article ) itsfoss: 21 Terminal Shortcuts by Sagar Sharma - Lastly, we cover Terminal Shortcuts. They might not seem like such a big deal but getting the hang of a couple shortcuts can make a big difference for your efficiency and speed leading to substantial quality of life improvements.

( Video ) Unix and Linux History by Jon "maddog" Hall - Jon Hall gives a great rundown on Linux and Unix history in the 50 year period from 1969 to 2019. It's an informal recount mixed with personal experiences and that's exactly what makes it such a compelling watch.

( Video ) The Rise of Unix. The Seeds of its Fall. by Asianometry - A great brief 16-minute video that covers the rise of Unix. Started in the legendary Bell Labs by Ken Thompson and Dennis Ritchie. As a personal note, if you're interested in video essays on technology, economics, business and politics I would highly recommend you explore more of Asianometry's videos.

( Article ) The Early Days of Linux by Lars Wirzenius - In keeping with the theme of historical reading the following article goes over some of the very first days of Linux. Lars witnessed the birth of Linux and he also helped fix some of its first issues.

( Short Rant )✨How does the Unix Philosophy matter in modern times? by dlarge65 - There are many diverging opinions on the Unix philosophy. Some say it is over-idealized and irrelevant in the modern day, others argue that there was maybe a philosophy at one point but practicality won over and so on. While I'm still very much new to Linux I enjoyed this rant which takes on a more balanced stance and viewpoint on the Unix Philosophy. It is too early to give my thoughts on the matter but moving forward I plan on taking the balanced approach.

Section Projects

Project 2 - Your First Wargame -> Over The Wire: Bandit

OTW-Bandit-L0.png

Wargame vs CTFs
If you're like me, you probably thought that the term wargame and CTF were interchangeable but they are 2 different things although they are related.
To Summarize: Capture The Flag (CTF) - Time Limited / Wargame - Not Time Limited
Capture The Flag (CTF) - CTFs are exercises where participants try to find strings, called "flags" which are hidden inside purposefully vulnerable programs or websites,
Wargame - A cyber-security challenge where competitors must exploit or defend a vulnerability in a system or application to gain or prevent access to a computer system.

In essence CTF's are time-based competitions where people try to come out on top, whereas wargames aren't really time limited or competitions, although they do have CTF style challenges.

For this project your job is to go through The Beginner Wargame - Bandit

Why a wargame?
First off, they are very fun! I was always a bit hesitant to read through man pages, but I found myself going through documentation and scouring man pages all to get that next flag. 
Secondly, they can be very challenging. You'll be forced outside of your comfort zone quite often. Bandit will lead you to explore  concepts in networking, compression, decompression, hex dumps, the shell and more.

Bandit
Bandit is a wargame aimed at beginners. Wargames are usually split into levels. We start at Level 0 and try to reach the end.

Why Bandit?
I was pretty hesitant about doing CTFs, they tend to have a heavy cybersecurity focus, which is important, but at our stage can be a distraction.
However, bandit is the ultimate Linux Lab.
- You will use a variety of new flags for commands you commonly use
- You will get a better grasp on user and file permissions
- You will become more comfortable google searching for Linux information
- You will be forced to read the man pages
- You will cover Networking Topics, Data encoding, compression and decompression
- You will be introduced to git, and its commands
- You will go over shell and CLI behavior

After doing Bandit, I am much more comfortable reading documentation, and I have a better idea on the skills I'm missing as a Linux user. You'll learn a lot doing your first wargame and I hope you'll have as much fun working through it as I did.

Bandit Level - 0
Linux Fundamentals

Linux Journey

Course Overview

Goals of this Course

Course notes

Required Reading

Section Projects

Project -
Linux Fundamentals

Learning the bash Shell

Linux Fundamentals

UNIX and Linux System Administration Handbook

Linux Fundamentals

The Debian Admin's Handbook

Linux Fundamentals

Linux Fundamentals Capstone Project

Networking Fundamentals

Networking Fundamentals

Intro to Computer Networks

Networking Fundamentals

Cisco CCNA

Networking Fundamentals

Network Warrior

Networking Fundamentals

Practical Packet Analysis

Networking Fundamentals

Core 5G and Beyond

Networking Fundamentals

Networking Fundamentals Capstone Project

Intro to Programming

Intro to Programming

Python Programming

Intro to Programming

Automate the Boring Stuff with Python (Optional)

Intro to Programming

Intro to MySQL

Intro to Programming

Git

Intro to Programming

Intro to Programming Capstone