Linux Fundamentals

The Linux Upskills Challenge

LinuxUpskillChallengeTitle.png

Course Overview

We begin our journey into Systems and Programming with the Linux Upskill Challenge  ( GitHub )  

This challenge is a great way to take our first steps into Linux. It is a month long commitment that you can follow at your own pace, but if you dedicate 2-3 each day it will take you roughly 21 days to complete the challenge.
For our purposes we will be using the Challenge to as a way to learn Linux for the first time and to also begin building on our Linux Command Line Skills.

Goals of this Course
Course Notes

The Linux Upskill Challenge has a lot of great concise and detailed explanations for all of the topics they cover. Which makes it pretty hard to make any notes that wouldn't just be a copy & paste.
As a middle ground the following notes are commands and information I found particularly useful.

A quick note for people who are windows users!

Some of you may be using WSL to access your cloud instance. You might experience clock-drift on your device making you unable to access to the cloud CLI. If this happens try running the following command in the block below. 
This should serve as a temporary fix to your clock-drift but I would recommend looking online for a more permanent solution.

sudo hwclock -s

The ls command and its switches
    - ( -l ) : lists directory contents
    - ( -a ) : lists hidden files
    - ( -t ) : lists by date created
    - ( -r ) : lists in reverse order
Some notes on Sudo & Root Best practices 
    - Don't leave root open
    - Be careful of using sudo to open GUI applications, if you have to, use sudo -h 
    - /etc/shadow : A very important space that stores information about the user passwords on a Linux system
    - Hackers can't auto-crack if root is locked down
Commands used to find files
    - locate | find | grep | which
Ownership and Permissions in Linux
    - ls -ltr is a good way of showing who owns what files and what permissions they have
    - chmod
         - u = user | g = group | o = others | a = all
         - ( + ) = add permission | ( - ) = remove permission | ( = ) = set permission
         - r = read | w = write | x = execute
         - u = user | g = group | o = others | a = all
         - 1 = execute | 2 = write | 4 = read
examples:
         - chmod u+w hello.txt | Gives user write permissions to the hello.txt file
         - chmod 777 hello.txt | Gives read write and execute permissions to everyone
Important File Transfer protocols
Linux can share files through a variety of protocols. These 3 are key protocols to keep in mind
         - SMB: Microsoft file sharing, useful on a local network of Windows Machines
         - AFP: Apples file sharing, useful on a local network of Apple Machines
         - SFTP: Access & transfer files over SSH
You can get practice with vim by running vimtutor
Turn a file into a bash script with #!/bin/bash

Required Reading

The required reading section in the Linux Upskill Challenge covers all the material you need to get started. I added setting up SSH login as required since the challenge lists it as an extension.

Similar to before, the recommended readings posted on the challenge are very good and definitely worth going through. The 2 articles below are extensions that I found particularly interesting or practical.

Section Projects

Project 1 - Take your first steps into the world of Self-Hosting

What is Self-hosting? 
Self-hosting is the practice of running and maintaining a website or service using a private web server, instead of using a service outside of someone's own control.

We should now be knowledgeable enough about Linux to comfortably follow tutorials on self hosting.
The world of self hosting is a rabbit hole and there are more services out there than you will ever have time to understand and use, which also means that there's bound to be something for everyone.

For this project your task will be to self host 3 services

The first 4 services I listed are the ones I first set up when I finished the challenge. WordPress and Pi-hole were relatively straightforward but getting Grafana & Prometheus to work was definitely a challenge.
I also linked to the Awesome Selfhosted GitHub page that shows 100's of services you can self host.

Learning the Unix OS

LearningTheUnixOS_5th_Edition.jpgLearning the Unix OS - 5th Edition
Jerry Peek, Grace Todino & John Strang ( O'Reilly, 2002 )

This book is now over 20 years old, yet it has aged gracefully.
Roughly 90% of the information inside is still relevant today and the other 10% can be treated as a history lesson. 
The reason can be found in the book's preface.

This book teaches basic system utility commands to get you started with Unix... We cover a commands most useful features...

And it more than lives up to its purpose. While Linux & Unix have changed a lot over the years the command line itself hasn't changed radically. The reason why the commands shown are interchangeable is that many of the core commands are identical in terms of syntax and functionality between Unix & Linux. There are of course some differences, but they are not enough to be a problem at our current level. We won't learn about Modern Filesystems or Containers, but we will learn enough to get even more confident with working through Linux.

Book Overview

Our next step in the systems fundamentals course will be Learning The Unix OS by O'Reilly. You may treat this book as a Practical Guide to Unix. This book is 157 pages long and will take you 7 days to finish if you dedicate between 2-3 hours every day to work through it.

When you look at the title you will notice the word Unix, but don't let the word Unix discourage you. Much of what we will be going over will be regarding the Command Line Interface. Which for our purposes is practically interchangeable with Linux. Don't misunderstand, they are two different operating systems, with the key difference being that Linux is Open-Source while Unix is Proprietary

The main goal of this book is to serve as a Practical Starter Guide, covering the main essentials of the CLI and Unix OS. That is why this book is still remains relevant even today. While we did go through an introductory course already, that being the Linux Upskill Challenge, there are some nuances that that this book goes through that will be very helpful to us.
We'll explore the various types of commands, Windows Systems, basic File Management, I/O-Redirection and even some interesting pieces of Linux history. When I finished going through this book, I felt confident that I could start working on this roadmap and bring something interesting to the table. Although we still have a long way to go before we can call ourselves true adepts, but steadily, we are making progress.

✨Personal Note

( The following has been added to the overview page. But I am keeping this here as this was when I thought of the idea. )
Throughout this journey I will do my best to be as objective as I can. However, I am still in the middle of my learning and I am only human. While technology itself is neutral, the way we use it, how we think about it, what we think is best moving forward and so on, are things that are human viewpoints. There are of course times when things are simply 0 or 1, but as you go through your own journey you will find that you will have to come up with your own definitions and reasons for doing things backed by personal preference or well researched analysis or simply that's the way things are done. 
So, to carry on with that spirit I will place the sparkling star icon✨ whenever certain notes or topics are things I am speaking my own mind on. I encourage you to come up with your own analysis or thoughts on those topics.

Goals of this Book
Course Notes

I quite enjoyed making the notes for this book since it led me to making several web searches on various different topics about Linux both old and new. 

Required Reading

Our required reading for this section will be a mix of reviewing old commands with learning new ones. We begin by covering some particularly useful List and System/Network monitoring commands. For those I recommend using the man page in order to get more comfortable with using the man command. After that we'll cover some articles on the Digital Ocean and RedHat community blogs that cover some more useful commands.

✨Finally we end with a more philosophical article, technical ability is important but I encourage you to consider the abstract side as well. Primarily questions like: "Why is Unix & Linux designed this way?" "What the point of FOSS?" and musings of that nature. 

( Self-Guided ) - Look into the following commands and go through them by using the man command. Read the Name, Synopsis, Description and go over some of the Options, then try the commands out. For system memory commands I recommend paying attention to the -h flag which shows displays the output in a human readable format. After you read the command's man page if it's use-case or meaning is unknown to you, do a web-search for more information.

( Article ) Digital Ocean Community: Top 50+ Linux Commands by e...f - By this point we have gone over most of the commands listed in the Digital Ocean article. I recommend reviewing any commands you're not confident in and pick up any that you may not know.

( Article ) RedHat - Enable Sysadmin: Linux Commands: du... by Tyler Carrigan - The du command is an excellent tool that will give you insight into how your storage is being used. This short article will go over the most used flags and how to make the most of them. 

( Article ) RedHat - Enable Sysadmin: How to Manage Linux Permissions... by Damon Garn - Managing user permissions is an essential task for any sysadmin. User access control is an essential security feature meant to protect data by giving only certain users access to data and resources based on certain contexts. If you still feel unconfident about managing permissions focus on this post and make sure to understand absolute/octal mode.

( Article ) Linux-Databook: The Unix and Linux Philosophy by dboth - Every Operating System has it's own philosophy. To steal the quote listed in the post. 

An operating system, by its nature, embodies the philosophy of its creators…"  - Mike Gancarz

Essentially any OS from Linux to Windows to macOS has their own way of doing things. It's Linux's philosophy that makes it so powerful in a capable users hand. As always I recommend you take away your own conclusions from this article. You may agree with much of it now and maybe come to disagree with much of it later and vice versa, even I don't agree with every tenet.

( Article ) itsfoss: 21 Terminal Shortcuts by Sagar Sharma - Lastly, we cover Terminal Shortcuts. They might not seem like such a big deal but getting the hang of a couple shortcuts can make a big difference for your efficiency and speed leading to substantial quality of life improvements.

( Video ) Unix and Linux History by Jon "maddog" Hall - Jon Hall gives a great rundown on Linux and Unix history in the 50 year period from 1969 to 2019. It's an informal recount mixed with personal experiences and that's exactly what makes it such a compelling watch.

( Video ) The Rise of Unix. The Seeds of its Fall. by Asianometry - A great brief 16-minute video that covers the rise of Unix. Started in the legendary Bell Labs by Ken Thompson and Dennis Ritchie. As a personal note, if you're interested in video essays on technology, economics, business and politics I would highly recommend you explore more of Asianometry's videos.

( Article ) The Early Days of Linux by Lars Wirzenius - In keeping with the theme of historical reading the following article goes over some of the very first days of Linux. Lars witnessed the birth of Linux and he also helped fix some of its first issues.

( Short Rant )✨How does the Unix Philosophy matter in modern times? by dlarge65 - There are many diverging opinions on the Unix philosophy. Some say it is over-idealized and irrelevant in the modern day, others argue that there was maybe a philosophy at one point but practicality won over and so on. While I'm still very much new to Linux I enjoyed this rant which takes on a more balanced stance and viewpoint on the Unix Philosophy. It is too early to give my thoughts on the matter but moving forward I plan on taking the balanced approach.

Section Projects

Project 2 - Your First Wargame -> Over The Wire: Bandit

OTW-Bandit-L0.png

Wargame vs CTFs
If you're like me, you probably thought that the term wargame and CTF were interchangeable but they are 2 different things although they are related.
To Summarize: Capture The Flag (CTF) - Time Limited / Wargame - Not Time Limited
Capture The Flag (CTF) - CTFs are exercises where participants try to find strings, called "flags" which are hidden inside purposefully vulnerable programs or websites,
Wargame - A cyber-security challenge where competitors must exploit or defend a vulnerability in a system or application to gain or prevent access to a computer system.

In essence CTF's are time-based competitions where people try to come out on top, whereas wargames aren't really time limited or competitions, although they do have CTF style challenges.

For this project your job is to go through The Beginner Wargame - Bandit

Why a wargame?
First off, they are very fun! I was always a bit hesitant to read through man pages, but I found myself going through documentation and scouring man pages all to get that next flag. 
Secondly, they can be very challenging. You'll be forced outside of your comfort zone quite often. Bandit will lead you to explore  concepts in networking, compression, decompression, hex dumps, the shell and more.

Bandit
Bandit is a wargame aimed at beginners. Wargames are usually split into levels. We start at Level 0 and try to reach the end.

Why Bandit?
I was pretty hesitant about doing CTFs, they tend to have a heavy cybersecurity focus, which is important, but at our stage can be a distraction.
However, bandit is the ultimate Linux Lab.
- You will use a variety of new flags for commands you commonly use
- You will get a better grasp on user and file permissions
- You will become more comfortable google searching for Linux information
- You will be forced to read the man pages
- You will cover Networking Topics, Data encoding, compression and decompression
- You will be introduced to git, and its commands
- You will go over shell and CLI behavior

After doing Bandit, I am much more comfortable reading documentation, and I have a better idea on the skills I'm missing as a Linux user. You'll learn a lot doing your first wargame and I hope you'll have as much fun working through it as I did.

Bandit Level - 0

Linux Journey

Course Overview

Goals of this Course

Course notes

Required Reading

Section Projects

Project -

Learning the bash Shell

UNIX and Linux System Administration Handbook

The Debian Admin's Handbook

Linux Fundamentals Capstone Project