Welcome to Session 2 of our Cyber Security Challenge Training and we re gonna capture some shells 🙂

Last Updated: 2024-01-02

Why

Linux Shell Basics, Tips and Tricks, to help you with CTFs and also #ACSC Challenges

FAQ

  1. What if I don't have a laptop where I can run ssh or its broken : we have a VM we can provide for you, please notify your instructor, see last section of this tutorial

What

We will cover the following topics:

How should I prepare

What you'll need

Desired Learning Outcome

What you'll learn

Homework or Preparation

Prepare

PreRead

The role of a shell in the linux OS

https://linux-kernel-labs.github.io/refs/heads/master/lectures/intro.html

An Operating system runs applications and the kernel translates to the hardware. A shell or terminal is a method of interacting with the entire operating system as a human without a graphical user interface (GUI). That means, that the shell is located in user space and the (human) agent uses the shell to interact with kernel space via so-called utilities or commands , (used here synonymously).

Instructions entered in response to the shell prompt have the following syntax:

command [arg1] [arg2] .. [argn]

The shell parses the words or tokens (commandname , options, filesnames[s]) and gets the kernel to execute the commands assuming the syntax is good.

Users in Linux /Unix

Unix is a multi user environment

$ whoami
croedig
$ id
uid=1000(coder) gid=1000(coder) groups=1000(coder),999(docker)
$ groups
coder docker
$ cat /etc/passwd
root:x:0:0:root:/root:/bin/bash
daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin
bin:x:2:2:bin:/bin:/usr/sbin/nologin

Let's talk about the last command:

  1. it shows you that there are a lot of non-human users. This has to do with the concept of Least Privilege.
  2. For each user , we see the following entries

How many other users are logged in (or how many terminals do I and them have open)?

$ who
croedig
$ su - croedig
$ 
$ sudo cat /etc/shadow
root:*:19702:0:99999:7:::
daemon:*:19702:0:99999:7:::
bin:*:19702:0:99999:7:::
$ sudo su -
#

Privileged users

On Linux, there is always a root user, almost exclusively with id=0 (the highest named user is 60000.

Root is all-powerful and it is impossible to restrict its access.

Another important concept is sudo (superuser do), where a command is executed by the normal user on behalf of the superuser.

Each process has 3 user ids:

"Real" id: the owning user

"Effective" id : determines privileges

"Saved" id: set by exec to match the effective id

Conversely, each file is owned by 1 user id !

However, the god-like privileges of root are sub-structured into so-called capabilities, you can find your current capabilities using capsh. https://man7.org/linux/man-pages/man7/capabilities.7.html or https://0xn3va.gitbook.io/cheat-sheets/container/overview/basics 

$ cat /proc/$$/status | grep Cap
CapInh: 0000000000000000
CapPrm: 0000000000000000
CapEff: 0000000000000000
CapBnd: 00000000a80425fb
CapAmb: 0000000000000000
$ capsh --decode=00000000a80425fb
0x00000000a80425fb=cap_chown,cap_dac_override,cap_fowner,cap_fsetid,cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_net_bind_service,cap_net_raw,cap_sys_chroot,cap_mknod,cap_audit_write,cap_setfcap
$ getcap  -r / 2>/dev/null
will search for binaries with capabilities

Permissions

Under POSIX, any file (and everything is a file, after all) has three permissions set by default, user, group and others.

Owner Group Other

r w x r w x r w x

4 2 1 4 2 1 4 2 1

When a file is created its permissions are restricted by the umask of the process that created it.

These are the examples from the symbolic notation section given in octal notation:

Symbolic

notation

Numeric

notation

English

----------

0000

no permissions

-rwx------

0700

read, write, & execute only for owner

-rwxrwx---

0770

read, write, & execute for owner and group

-rwxrwxrwx

0777

read, write, & execute for owner, group and others

---x--x--x

0111

execute

--w--w--w-

0222

write

--wx-wx-wx

0333

write & execute

-r--r--r--

0444

read

-r-xr-xr-x

0555

read & execute

-rw-rw-rw-

0666

read & write

-rwxr-----

0740

owner can read, write, & execute; group can only read; others have no permissions

https://en.wikipedia.org/wiki/File-system_permissions

An important command here is the chmod, to modify the permissions:

$ chmod +x exe.sh
// makes it executable
$ stat .bashrc
  File: .bashrc
  Size: 3106            Blocks: 8          IO Block: 4096   regular file
Device: 10002eh/1048622d        Inode: 399403      Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2021-10-15 10:06:05.000000000 +0000
Modify: 2021-10-15 10:06:05.000000000 +0000
Change: 2024-01-15 12:32:45.847306257 +0000
 Birth: 2024-01-15 12:32:45.847306257 +0000
$ chmod 600 mykey
Sets a sensitive file to be rw only by the user
$ sudo usermod -aG docker coder

You also might find chown similarly useful.

Important for Security are very much the not so commonly known SETUID SETGUID and sticky bits, that let a program be executed under either user or group that is not itself.

Note the "s" in the permissions displayed below. This represents the SETUID bit and allows

Lets look at a famous example : ping, it requires to open a socket, which corresponds to the CAP_NET_RAW capability. Most systems wont allow an end-user to set capabilities, so instead the SETUID bit is set:

$ sudo apt update && sudo apt install inetutils-ping
...
$ which ping
ping: /usr/bin/ping
$ setcap cap_net_raw+p /usr/bin/ping
unable to set CAP_SETFCAP effective capability: Operation not permitted
$ stat /usr/bin/ping
  File: /usr/bin/ping
  Size: 78480           Blocks: 160        IO Block: 4096   regular file
Device: 10002eh/1048622d        Inode: 531677      Links: 1
Access: (4755/-rwsr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
Access: 2024-01-15 12:51:53.000000000 +0000
Modify: 2023-08-15 13:13:06.000000000 +0000
Change: 2024-01-15 12:51:53.867136654 +0000
 Birth: 2024-01-15 12:51:53.855136677 +0000
  File: /usr/bin/ping
$ cat /proc/$$/status | grep Cap

A word on $PATH

You need to tell your shell where your executables are to be searched for.

Lets say you want to find out which binary is executed when you type bash:

$ which bash
/bin/bash
$ whereis bash
bash: /bin/bash /usr/share/man/man1/bash.1

Maybe you have your own binaries lying around in custom locations (in the following example I want to use locally built nodejs modules), so you need to update the PATH where they will be search for:

$ echo PATH
/Users/croedig/Downloads/google-cloud-sdk/bin:/Users/croedig/.rd/bin:/usr/local/opt/gnupg@2.2/bin:/usr/local/bin:/usr/local/MacGPG2/bin:/Library/TeX/texbin:/Applications/VMware Fusion.app/Contents/Public:/Users/croedig/.orbstack/bin:/Users/croedig/go/bin
$ export PATH="./node_modules/.bin/:$PATH"

A process' environment

Last, we should talk quickly about ENVIRONMENT: each process has attached to it an ASCII table of key=value pairs that describe its settings. These are also stored in address space and when we create a process from a parent process, the env will be the same (in the beginning). These values can include very juicy details that give you clues about the process and maybe even passwords or connection strings.

$ env
TERM_PROGRAM=Apple_Terminal
SHELL=/bin/zsh
P9K_SSH=0
ZSH=/Users/croedig/.oh-my-zsh
GOPATH=/Users/croedig/go
GOROOT=/usr/local/go
LC_CTYPE=UTF-8
REDIS_SERVER=127.0.0.1:6543

As users, we may be interested in what is going on between an application and the OS, or simply what is generally happening all over the OS. To this end, there are a plethora of utilities for us to find, inspect, trace and interact with the Linux OS.

Shell types

Shells are ways to interact with processes. At system boot, there is just a single process (init). It subsequently gets forked until there are many processes, some running in kernel mode and some in user land.

There is a convention that a process expects to inherit (at each fork) from its parents two open file descriptors, 0 and 1.

In modern OS, we use terminal emulators and you can find them under
/dev/pts (pseudo terminal slaves0

/dev/tty (controlling terminal file)

/dev/ttyN (virtual console terminal file)

https://www.youtube.com/watch?v=07Q9oqNLXB4

Linux:

  1. Bash
    " bourne again shell" . Today probably the most commonly used in Linux, it is a user friendly shell with history, customization, tab-completion, syntax highlighting and many more features that make it faster to work on the command line.

  1. Sh
    "bourne shell". The original, having a comeback these days in lightweight environments such as containers. It isn't particularly user friendly as it has the fewest features of all listed here. But you can get the same work done, might just take a bit longer.
  2. Zsh
    An extension of Bash and the leading MacOS shell, has more productivity features on top of Bash and a customization framework (oh-my-zshell)

  1. Tcsh and Csh
    A bit dated nowadays, its syntax was modelled after the C language

  1. Ksh, dash
    Specific distros (dash = Debian almquist shell) or niche systems (IBM AIX uses the Korn shell still) or other Unix-like Operating Systems (like Solaris has a lot of other options for shells) have specific shell implementations, those differ generally in their performance, individual features, user-friendliness and security (yes, shells can have vulnerabilities, too).

Changing shells

Should you encounter a "shell-script" (something, with a .sh extension or something that you identify as a shell syntax) but that somehow doesn't work in bash but has no shebang (!#/bin/sh or #!/usr/bin/env bash) you should check if it is was written for a different dialect of shell.

List your available shells and then change to zsh

$ cat /etc/shells
# List of acceptable shells for chpass(1).
# Ftpd will not allow users to connect who are not using
# one of these shells.
/bin/bash
/bin/csh
/bin/dash
/bin/ksh
/bin/sh
/bin/tcsh
/bin/zsh
$ chsh -s /bin/zsh

When you want the output from the commands run in a shell without affecting your current shell, you can spawn a sub-shell:

Using sub-shells (uses brackets)

$ export pid=$(pidof myprocess) 
$ (cd /; ls -la;)

This is also used often for spawning processes to be run in the background (using the " & " operator at the end of the command)

MacOS:

  1. Zsh
    As mentioned above, MacOS has deprecated bash in favour of zsh
    Mostly, the general logic of shells is the same as for Linux, since MacOS (Darwin) belongs to the unix-family, however, implementations of utilities and the OS itself differs from Linux, so careful: it might not work the same way or you have to adapt the syntax.

Windows:

  1. Powershell
    Is a cross-platform automation tool using object-oriented syntax and extremely powerful query capabilities. It runs on all OS and is more like an SDK than a shell. If you have any choice whatsoever, use this one on Windows, not the other option:
  2. CMD
    The command shell is that old-fashioned black thing ... that only Windows people know what to do with... It only runs on Windows

Over time, we hope to update this with a selection of materials for you to use online/free.

Killercoda

Recommended labs for today

Please navigate to the Linux tile and select your favourite content. In general, you can do these labs in your own time, we will do one together today.

https://killercoda.com/pawelpiwosz/course/linuxFundamentals

Please select Lesson 2 : Your best friend - man

Man pages in linux are "manuals" and they are live savers, especially for commands that have obscure and endless options and parameters.

Here is a list of labs that can be consumed for free (sometimes after a signup)

An online set of challenges hosted on a VM that you ssh to from your machine. You will be executing the commands remote, so apart from ssh you don't need anything installed on your machine.

Optional: Increase Screen Zoom Level

Somebody noticed that it's easy to waste time if you misread characters (cough cough), maybe increase your zoom level to avoid this problem.

Getting started

Navigate to https://overthewire.org/wargames/bandit and start at Level0.

For your increased productivity

Favourites

Xargs and Find: a match made in heaven

xargs

is a command line utility that executes whatever you feed it from STOUT

find

can literally find anything, it is extremely powerful

find . -type d -empty | xargs rmdir
find . -type f -name "*.txt" | xargs -I{} mv {} {}.bak

The first command removes all empty directories, the second renames all .txt files to .bak files ( the curly brackets are placeholders).

One note on piped commands: they will break at an error (not continue after they throw an error), so that can be annoying. In that case you could also combine find with exec.

Read this detailed explanation here https://www.everythingcli.org/find-exec-vs-find-xargs/

Quickhands

// bring back the historical command (4 commands ago in this example)
$ !-4
// ctrl +R opens the history search
$ 
bck-i-search: _  
// go back to previous directory (works like a toggle)
$ cd -
// repeat previous command
$ !!
// recursive wildcard (depends on shell, see screenshot)
$ ls -l examples/**/secret*/**/
// retrieve the PID of my current process
$ echo $$

A couple notable utilities

$ tree 
$ touch
$ lsblk lsns 
$ pidof

Networking: what is going on on ports

$ sudo lsof -i -P -n | grep LISTEN
$ sudo netstat -tulpn | grep LISTEN
$ sudo ss -tulpn | grep LISTEN
$ sudo lsof -i:22 ## see a specific port such as 22 ##
$ sudo nmap -sTU -O IP-address-Here

And a personal favourite (written by Jess Frazelle) https://github.com/genuinetools/amicontained that tells me A LOT about where I am running on (even if I m not in a container).

$ wget amicontained https://github.com/genuinetools/amicontained/releases/download/v0.4.9/amicontained-linux-amd64; chmod +x amicontained-linux-amd64; ./amicontained-linux-amd64
Container Runtime: docker
Has Namespaces:
        pid: true
        user: false
AppArmor Profile: cri-containerd.apparmor.d (enforce)
Capabilities:
        BOUNDING -> chown dac_override fowner fsetid kill setgid setuid setpcap net_bind_service net_raw sys_chroot mknod audit_write setfcap
Seccomp: disabled
Blocked Syscalls (28):
        MSGRCV SYSLOG SETUID SETGID SETSID SETREUID SETREGID SETGROUPS SETRESUID SETRESGID VHANGUP PIVOT_ROOT ACCT SETTIMEOFDAY SWAPON SWAPOFF REBOOT SETHOSTNAME SETDOMAINNAME INIT_MODULE DELETE_MODULE FUTIMESAT UTIMENSAT FANOTIFY_INIT OPEN_BY_HANDLE_AT FINIT_MODULE KEXEC_FILE_LOAD BPF
Looking for Docker.sock

More materials

Search on github for CTF relevant gists, like these

https://gist.github.com/FrankSpierings/4af182c7c3aee32796792d35bedbe0fb

Knowing thy system

What else should / could I find out from my shell?

What is this shell running on? Where ? How contained? Virtualized? On-prem or in the cloud?

In real life, it can be pretty important to understand the periphery or the surrounding of my shell.

Let's do some recon

sudo apt-get install dmidecode

dmidecode -s system-manufacturer

sudo dmesg | grep "Hypervisor detected"

Aha , so this particular shell runs on a VM

A quick google on "openstack" tells us that this is likely a private cloud ( advanced users would use this recon intelligence to search for a standard architecture of Openstack and for known exploits or weaknesses)

But on another shell you might get this instead

So we're inside an Ubuntu container, which I would follow-up with a dump of

$ env

$ uname -a

$ cat /etc/os-release

$ lsb_release -a

Which tells me that I'm on a container orchestration system called kubernetes, ( which again has a reference architecture and known weaknesses): Also, the fact that I'm in a container, tells us that it will likely by minimized i.e. ship very few utilities. For containers , the amicontained utility gives us lots of info.

Additionally, I can infer the layout of a monitoring design of the system from knowing if I'm dealing with a cloud or a traditional deployment.

Topic observability

Depending on what you are doing, you might want to be „quiet" on a system, so as to not alert a Soc team of your presence (there are types of purple team exercises that test if an adversary could be hiding within the system without detection) . Generally speaking network enumeration or wide scanning is very noisy and suspicious.

This is a vast and advanced topic, but it bears mentioning that it is generally very useful to check what type of host I'm executing something on.

Exercise for the bandit game:

Draw a small diagram of how your network connection looks like: protocol, ports, hostname, operating system, how many users are there, what rights do I have. Is this connection encrypted ?

Advanced exercise:

Sniff your own traffic ( wireshark or similar) and find the connection: what do you see there?

( this exercise will very likely be explained in the ‚network basics‘ ) lab

Fear not, we got you covered

Get a cloud VM

Once you ve been given credentials, please navigate to Sign in to Coder

Login and then press this button (or select "Create New Workspace with the "Robot" logo)

Once your workplace is provisioned, please press the Terminal button on the right, and a webterminal will appear from which you can connect to the bandit game.

Congratulations, you've successfully completed this training

What's next?

Further reading

Reference docs