Security, Protection and Case Studies

Goals of OS Security

• Confidentiality — data is only readable by authorized users.
• Integrity — data is not altered by unauthorized users.
• Availability — the system is up and responsive.
• Accountability — actions can be traced to their actors.

Authentication

Something you know (password), have (smart card, token), or are (fingerprint, face). Strong systems combine two or more factors (MFA). Passwords are stored as salted hashes (bcrypt, argon2).

Access Control

Once a user is authenticated, access control decides what they can do:

• Discretionary Access Control (DAC) — owner sets permissions (UNIX rwx).
• Mandatory Access Control (MAC) — system policy (SELinux, AppArmor).
• Role-Based Access Control (RBAC) — permissions tied to roles.

Protection Mechanisms

• User vs kernel mode.
• Virtual memory protects process address spaces.
• File system permissions and ACLs.
• Firewalls, SELinux, secure boot, disk encryption.

Threats

• Malware — virus, worm, trojan, ransomware, rootkit, spyware.
• Buffer overflow — overwrite return address to hijack execution. Mitigated by DEP, ASLR, stack canaries.
• Privilege escalation.
• Denial of service.
• Side-channel attacks (Meltdown, Spectre).

Defense in Depth

No single measure is enough. Combine strong authentication, least privilege, patching, monitoring, backups, and user education.

Cryptography in the OS

• Password hashing.
• Full-disk encryption (BitLocker, LUKS, FileVault).
• TLS for network services.
• Signed kernel modules.

Case Study: Linux

Linux is a free, open-source, monolithic-with-modules kernel released by Linus Torvalds in 1991.

• Architecture — monolithic kernel, loadable modules, POSIX system-call interface.
• Processes — scheduled by the Completely Fair Scheduler (CFS).
• Memory — demand paging with multi-level page tables, LRU-approximation replacement.
• File systems — ext4, XFS, Btrfs, F2FS over a virtual file system (VFS) layer.
• Security — DAC by default, SELinux or AppArmor for MAC, cgroups + namespaces enabling containers.
• Distributions — Debian, Ubuntu, Fedora, Red Hat, Arch.

Case Study: Windows

Windows NT/10/11 is a hybrid-kernel OS by Microsoft.

• Architecture — hybrid kernel: executive services + microkernel core + HAL.
• Processes and threads — priority-based preemptive scheduling.
• Memory — demand paging, working-set trimming.
• File systems — NTFS, ReFS, FAT, exFAT.
• Security — ACLs on every object, User Account Control (UAC), Windows Defender, BitLocker.
• Driver model — WDM and WDF; signed drivers required.

Case Study: Android

Android is a mobile OS built on a customized Linux kernel with a managed runtime (ART). Each app runs in its own Linux UID for isolation. SELinux enforces mandatory access. The Play Store and Google Play Protect screen apps for malware.

Virtualization and Containers

Hypervisors (Xen, KVM, VMware ESXi, Hyper-V) let multiple OSes share one machine. Containers (Docker, LXC) virtualize only the user space on a shared kernel using namespaces and cgroups; lighter weight, faster to start.

Summary

OS security layers authentication, access control, protection, and cryptography to defend confidentiality, integrity, and availability. Linux, Windows, Android, and hypervisors are the dominant platforms today and each illustrates how OS theory translates into real systems.

Important Questions

1. State the four goals of OS security.
2. Differentiate DAC, MAC, and RBAC.
3. List five types of malware.
4. Explain buffer overflow and three countermeasures.
5. Describe the Linux kernel architecture.
6. List the main subsystems of Windows NT.
7. Differentiate virtual machines and containers.
8. How is Android security enforced?