There are two major threads in "real world" operating systems, the unix thread and the windows thread. People who work for IBM might take issue with this and point out that IBM mainframe operating systems are just as important, but we will not be talking much about these in this course. Unix systems have traditionally been open source (but not always), while the windows systems are proprietary, with their internal working kept secret.
. The first operating system that could properly be called a Unix system was developed by Ken Thompson at Bell Labs in the early 1970's. It originally ran on the DEC PDP-7 minicomputer, but was quickly ported to the more modern PDP-11. Ken Thompson had not started from scratch; he borrowed many ideas from an earlier research operating system called Multics.
Porting an Operating System from one hardware architecture to another is a lot of work, so one of the ideas that he developed for unix was to write virtually all of the system in C (C was also developed at Bell Labs around the same time). This meant that once a C compiler had been written for a new hardware architecture, it was fairly simple to port Unix to the new architecture by simply (Hah!) recompiling it. Only a small amount of code such as device drivers and interrupt handlers had to be written in assembly code.
In the 1980's, two strands of Unix developed. One was System V, a commercial product from AT&T. (Aside: For most of the 20th century, AT&T was a legal monopoly; in return for being able to run the phone system without competition, they were forbidden from entering other commercial areas. In 1984, AT&T's monopoly on local phone service was broken up, but this permitted AT&T to go into new businesses such as computer manufacture and software).
The second strand of Unix was Berkeley Unix (Berkeley Software Distribution or BSD). This was rework of the original Unix. The head software architect for this was Bill Joy, then a graduate student at UC Berkeley. He later went on to start Sun Microsystems.
Numerous versions of BSD were released, initially all running on Digital Equipment (DEC) hardware, PDP-11's and then VAX. The most important release was BSD-4.2 (1983). This release included the first implementations of many operating features that are still standard, including the socket interface, TCP/IP (the protocol of the internet), signals, sendmail (an email client), as well as the BSD Unix mascot, a daemon.
Numerous other flavors of Unix developed over time. Here is a chart that shows the interconnection of the various flavors of Unix.
In the 1980s, Project Athena at MIT developed a number of tools and applications for Unix systems, some of which are still in use today. This was funded by the two computer heavyweights of the time, IBM and DEC. The orientation was on networked and distributed computing at the time when most computing was done locally.
One of the most important developments of Athena was the concept of a distributed file system, in which users could log into any computer on the network and have access to their files and other shared resources such as printers and shared programs. Here are two of the applications:
In 1991, Linus Torvalds, a student at the University of Helsinki in
Finland, started work on a non-commercial version of Unix called
Linux. This has become a wildly successful operating system, and there are
now numerous different flavors of Linux, mostly open source and freely available.
Linux has even been adopted by IBM, which for years was philosophically opposed to
any kind of open source or free software.
Tux, the Linux mascot
In addition to the Unix kernel being open source and freely downloadable, there are a huge number of open source and free user applications, mostly available from the Gnu (Gnu's not Unix) project, part of the Free Software Foundation. This includes the bash shell, compilers such as gcc and g++, text editors such as emacs, and hundreds of other utilities.
Here is a high level schematic of the Unix Operating System
The Unix command line interface is called the shell. There are numerous versions of the shell which differ in minor ways, but the most widely used is called bash (the Bourne again shell). The shell runs as an application program in user space.
The shell displays a prompt and the user types commands. Commands can have arguments (retrieved by the program in the argument vector argv. The output of a command can be redirected to a file with the > character. If the command will be looking for input from the keyboard, input can be redirected to read from a file instead with the < character.
For example, the command ls displays the contents of a directory (Directory is the Unix term for a Windows folder.) This takes numerous arguments, one of which is -l (long) which means to display more information than just the file name, information like file size, owner, and permissions. Normally this command displays its output on the terminal, but this command redirects the output to a file called temp
ls -l > temp
It is also possible to redirect the output of one command as the input of another command. This is called piping. The pipe character is the |. For example, here is how you would sort the output of the ls -l command.
ls -l | sort
The original Microsoft Operating System was MS-DOS, in the 1980's. This was a simple OS with no security, no multiprocessing, and a command line interface.
Apple was making inroads into Microsoft's business with its GUI based systems, and so Microsoft put a GUI on top of MS-DOS. Windows 3.0 came out in 1990, Windows 3.1 (a dramatic improvement) in 1992. But all programs ran in the same address space.
Windows 95 (1995) contained most of the features of modern operating systems including virtual memory, process management, and multiprogramming. But there was still no security. Windows 95 was a commercial success, in spite of the fact that it crashed a lot.
By this time Microsoft had figured out that they could not just keep adding more features to MS-DOS, so they decided to build a new operating system from scratch. The result was Windows NT (New Technology). The API system was called WIN32.
Microsoft operating systems have to be backward compatible; that is, all of the applications that ran on the old Windows systems should still run on newer systems. This requires another layer in the API software to deal with this.
There were earlier versions of NT, but Windows 2000 was the first widely implemented system, and was the standard for several years, followed by Windows XP, and later Windows Vista.
Windows 2000 introduced Plug-and-Play (PnP) which allowed users to install new devices on the fly. Prior to this, users would sometimes have to manually set jumper switches or dip switches, and occasionally two (or more) devices would use the same interrupt handler address, and so one interrupt would trash the other. With PnP, the system can automatically detect new hardware, and automatically allocate whatever resources are needed (interrupt handler, DMA channels etc).
The original release of NT was about 3 million lines of C/C++ code. Windows Vista is more than 70 million lines of code. Part of this is due to the need for backward compatibility, nothing ever gets deleted, but most of the bloat is due to the addition of new features.
One feature of Microsoft operating systems is that many applications which run in user space on Unix systems are a part of the kernel in Windows. For example, X windows, the Unix GUI, runs as a user application but the GUI code for Windows is part of the kernel. Even Internet Explorer is considered to be a component of the Operating System.
There are multiple layers of system calls. At the core is the NT kernel ntoskrnl.exe, which has traditional system call interfaces, but these are not used by ordinary mortals, only by Microsoft programmers. There are a number of subsystems that run on top of NTOS. The only subsystem that we will use is Win32.
Other subsystems include POSIX and OS/2.
A key concept in NTOS programming is the object. An object can be a process, a thread, a file, a mutex, and so on. Each object has a handle which provides the means by which the object is manipulated by the user program. Each object also has a security descriptor which tells who can do what with it. These objects should not be confused with Object Oriented Programming Objects. Here are some examples
The kernel has a centralized object manager to keep track of all open objects. The root of the NT namespace is maintained in the kernel. Storage, such as file system volumes, is attached to the NT namespace. This namespace is constructed afresh whenever the system boots.
Windows can run on a number of different hardware architectures, not only different processors, but different support chipsets. In order to minimize the amount of hardware specific programming, the lowest layer of the OS is the Hardware Abstraction Layer (HAL). This layer translates all calls which deal directly with hardware into hardware accesses. Examples include the CPU itself, the Memory Management Unit, Interrupt controllers, physical devices, and the BIOS.
The registry is a special file system, (optimized for small files) attached to the NT namespace. The registry is organized into separate volumes called hives, which are like files. One of these hives is called SYSTEM. This is a file which keeps track of the system configuration. Other hives keep track of the configuration of installed software etc. Here is a list of some of the hives.
In Unix systems, generally each program consults a plain text resource file when it is opened. This contains all of the configuration information. For example, whenever you start a new instance of the bash shell, it consults a file called .bashrc in the user's home directory. This contains user preferences for such things as the prompt.
Modern operating systems can run in two (or more) modes, typically called user mode and kernel mode. An ordinary user who compiles and runs an ordinary program is in user mode, which means that the program only has access to its own memory space. However, most user programs need to access services provided by the kernel. An obvious example is reading from a file. While the typical user is not allowed access to the low level code that reads individual sectors, there needs to be some mechanism for user programs to access these services. These calls to kernel services are called system calls in the Unix world and Application Programmer Interfaces or APIs in the Microsoft world.
There are about 190 Unix system calls (the number varies somewhat from one system to another). They always take the form of a C function call; i.e. they take arguments which can be either ordinary variables or pointers to ordinary variables and they return a value. The value returned is usually of type int. In general, a positive or zero return value indicates that the call was successful, and a negative return value indicates that the system call failed for some reason. If the system call failed, a global external int variable errno will be set, and your program can look at this value to determine why the system call failed.
Although errno is an int, its values have symbolic names. You need to look at the man pages for each system call to determine the meaning of these values. For example, the system call to open a file is open. On success, it returns a file descriptor which can be used by other system calls to access the file. However, there are numerous reasons why an attempt to open a file may fail. Here is part of the on-line man page for open.
ERRORS
The named file is opened unless:
[ENOTDIR] A component of the path prefix is not a directory.
[ENAMETOOLONG] A component of a pathname exceeded 255 characters, or
an entire path name exceeded 1023 characters.
[ENOENT] O_CREAT is not set and the named file does not exist.
[ENOENT] A component of the path name that must exist does not
exist.
[EACCES] Search permission is denied for a component of the
path prefix.
[EACCES] The required permissions (for reading and/or writing)
are denied for the given flags.
[EACCES] O_CREAT is specified, the file does not exist, and the
directory in which it is to be created does not permit
writing.
[EISDIR] The named file is a directory, and the arguments spec-
ify it is to be opened for writing.
[EROFS] The named file resides on a read-only file system, and
the file is to be modified.
[EMFILE] The process has already reached its limit for open
file descriptors.
[ENFILE] The system file table is full.
[EEXIST] O_CREAT and O_EXCL were specified and the file exists.
Every possible error condition has a symbolic name (the words beginning with E in all caps) which are defined in a header file. The variable errno will be set on failure, and you can write code to find out what caused the error. Here is a skeleton of a C program that does this.
...
#include <errno.h>
extern int errno;
...
int main()
...
int returnval;
...
returnval = open(...)
if (returnval < 0) /* the open failed */
switch (errno) {
case EACCES: ..
case EDQUOT: ...
...
...
}
It is good programming practice to always check the return value of a system call that might fail (some system calls cannot fail), and take appropriate steps in the event of a failure. This will be required for all programming in this course.
Unix has a library function void perror(const char *msg) which displays your message along with a standard error message (based on the value of errno) on standard error.
Here is a code snippet showing how this might be used.
int fd;
fd = open( ... )
if (fd < 0) perror("Error opening file")
If the call to open failed because there was no file of that name, this message would be displayed on the terminal.
Error opening file: No such file or directory
There is also a function char *strerror(int errno) (make sure to include the header file string.h) which returns a string corresponding to the error number passed in as an argument.
Here is a list of some of the common Unix system calls.
Posix
Because of the proliferation of Unix operating systems, there have been a number of attempts to standardize Unix system calls. The most successful of these is POSIX (Portable Operating System Interface), an IEEE standard. There are a number of parts to Posix, POSIX.1 defines C programming interfaces to system calls.
For portability, Posix has defined a number of data types.
For example, the size of a file (an int on any system that
I know of) should be of type size_t. This may be confusing
at first, but you will get used to it.
Windows APIs
In the Microsoft world, a system call is referred to as an
Win32 API (Application Program Interface) (or a WIN64 API). If you
use any of these, you should include the header file windows.h.
This file contains definitions, macros, and structures for source
code.
As in Unix, the Win32 APIs have the same form as C function calls.
The number of Win32 APIs is much larger than the number of Unix system calls. One of the major reason why the number is so much larger is that all of the graphics calls which create and manipulate windows are part of the Win32 APIs, while the comparable Unix calls to XWindows run in user space. We will probably not be using any of the graphics calls in this course.
All versions of the Windows operating systems use the Win32 APIs, but each uses a slightly different subset. I will try to only include those APIs which are common across all versions of the Windows operating systems.
Hungarian Notation
If you read Microsoft C/C++ documentation or Microsoft sample code, you will need to understand Hungarian Notation. This is a method of naming variables which is a Microsoft convention. For example, the online help which describes the function ReadFile looks like this.
BOOL ReadFile(
HANDLE hFile,
LPVOID lpBuffer,
DWORD nNumberOfBytesToRead,
LPDWORD lpNumberOfBytesRead,
LPOVERLAPPED lpOverlapped
);
Hungarian Notation was developed by Charles
Simonyi - the original Microsoft chief software architect (he was
presumably of Hungarian extraction, although I have also read
that the term Hungarian Notation was sarcastic because it made
the code so hard to read). It is a naming convention that allows the
programmer to determine the type and use of an identifier (variable,
function, constant, etc.)
A variable name consists of an optional prefix, a tag which indicates the variable type, and the variable name. The prefix is lower case, the variable name itself starts with an upper case letter.
Common tags include
| Flag | Type | Example |
| b | Boolean | bGameOver |
| ch or c | single char | chGrade |
| dw | double word (32 bits) | dwBytesRead |
| n | integer | nStringLength |
| d | double precision real | dBalance |
| sz | null terminated char string | szLastName |
| p | pointer | pBuffer |
| lp | long pointer | lpBuffer |
| C | Class Name | CWidget |
In addition, most Microsoft sample code includes the header file
<windows.h> which redefines most data types.
You should never see a variable of type int in
Microsoft code. By convention, these are all uppercase.
Here are some examples.
| DWORD | unsigned long (stands for double word) |
| WORD | unsigned short (16 bits) |
| BOOL | boolean (Possible values are TRUE and FALSE) |
| BYTE | unsigned char |
| LPDWORD | pointer to a DWORD |
| LPVOID | Pointer to type void (a generic pointer) |
| LPCTSTR | Pointer to a const string |
Microsoft is migrating from WIN32 to WIN64, as processors move from 32 bit words to 64 bit words. This requires the introduction of new data types, such as POINTER_32, POINTER_64, INT32, INT64 (signed integers), DWORD32, DWORD64 (unsigned integers).
There are a number of conventions that apply to Win32 APIs.
Most kernel resources are objects, and are referred to with an opaque data type called a HANDLE. You can think of a handle as a pointer to the object, but you do not know the members of the object (that is what the term Opaque Data Type means). Some APIs will return a handle, and you can then pass that handle to other APIs. For example, the API which opens a file returns a handle and you can then pass that handle to an API which reads from that file.
The return types of Win32 APIs are more variable than Unix system
calls. Some functions return a BOOL (TRUE if successful, FALSE if it failed). Some will return a HANDLE. If the API fails, you can check
this by comparing the return value to INVALID_HANDLE_VALUE.
There is a function GetLastError() which returns the error
code indicating why it failed.
For example, the Win32 API which opens a file is CreateFile(...)
and it returns a handle.
Here is some skeleton code to demonstrate this.
HANDLE h;
h = CreateFile(...
if (h == INVALID_HANDLE_VALUE) {
printf("Error, could not open the file, error: %d\n",
GetLastError());
...
Displaying an error message on the screen which tells why
the error occurred is a little tricky. Here is a function
which does this, but it is far beyond the scope of this course
to explain it. Your Windows programs should call this function
when a call to an API fails. It returns an explanation of the
error.
char *GetErrorMessage()
{
char *ErrMsg;
FormatMessage(
FORMAT_MESSAGE_ALLOCATE_BUFFER |
FORMAT_MESSAGE_FROM_SYSTEM |
FORMAT_MESSAGE_IGNORE_INSERTS,
NULL,
GetLastError(),
MAKELANGID(LANG_NEUTRAL, SUBLANG_DEFAULT), // Default language
(LPTSTR) &ErrMsg,
0,
NULL
);
return ErrMsg;
}
You can download this file so you don't need
to retype it.
Here are some samples of Win32 APIs