Syllabus for

Harvard Extension School CSCI E-92 (formerly CSCI E-251)

Principles of Operating Systems (24794)

Spring 2017
Site last revised 7:30 PM 8-May-2017

Dr. James L. Frankel

 

After the current Spring 2017 semester, the next time CSCI E-92 will be offered is in the Fall 2018 semester.

Final Project Presentation Slots:
Time Slot Available
6:30 PM - 6:45 PM Slot 1 Filled by Derek
6:45 PM - 7:00 PM Slot 2 Filled by Per
7:00 PM - 7:15 PM Slot 3 Filled by Stephen
7:15 PM - 7:30 PM Slot 4 Filled by Kevin T.
7:30 PM - 7:45 PM Slot 5 Filled by Byron
7:45 PM - 8:00 PM Slot 6 Filled by Brendon
8:00 PM - 8:15 PM Slot 7 Filled by Mark W.
8:15 PM - 8:30 PM Slot 8 Filled by Vivek
8:30 PM - 8:45 PM Slot 9 Filled by Rob
8:45 PM - 9:00 PM Slot 10 Filled by Kevin F.
9:00 PM - 9:15 PM Slot 11 Filled by Jyot
9:15 PM - 9:30 PM Slot 12 Filled by Bennett

 

Quick Links:

Tuesdays 7:40-10:15 PM in 53 Church Street, Room L01.

Distance Learning Links:

During Class:

The class live video stream is available through the class Canvas web site under Lecture Video, Lecture Video (direct Matterhorn link), or Lecture Video (Zoom link). For Zoom, in the top right of the screen, click the link "Join a Meeting," and enter the meeting number for the course: 213 516 108.

Chat room (during class meeting)

After Class:

Videos of class and section are available on the course's Canvas web site under Lecture Video.

Midterm Exam

Our midterm exam is a three hour long proctored exam. Students who live within the six-state New England area (Connecticut, Maine, Massachusetts, New Hampshire, Rhode Island, and Vermont) must come to campus to take the exam (of course, any student may choose to come to campus to take the exam). Students who do not take the exam on campus in our classroom are required to procure the services of a Harvard-approved proctor as described in https://www.extension.harvard.edu/resources-policies/exams-grades-transcripts/exams-online-courses. That document states that the exam must be taken "within a specific 24-hour period." In our case, the 24-hour period starts on the date and time of our in-class exam and ends 24 hours later. That is, the exam must be started within that 24-hour period.

The exam allows open-book access to *only* the required textbook: Modern Operating Systems, Fourth Edition; Andrew S. Tanenbaum and Herbert Bos. No notes are allowed. No electronic devices are allowed.

Prerequisites:

Knowledge of data structures and programming experience CSCI E-22 (formerly CSCI E-119) (Data Structures), experience using UNIX, or equivalent.

Brief abstract:

This course examines the role of operating systems: process synchronization and scheduling; memory management including virtual memory, swapping, paging, and segmentation; file management; protection and security; input/output techniques, buffering, and resource allocation; deadlock detection and avoidance; system modeling; performance measurement and evaluation; and operating system case studies. An extensive lab project will be required of all students.

4 credits. Noncredit, undergraduate, and graduate credit.

Overview:

Computer Science E-92 is a detailed exploration of the internal algorithms and design of operating systems. Students are expected to be comfortable with designing, coding, and debugging programs of reasonable complexity while employing good programming style and structured techniques. In particular, knowledge of C, terminal and text file I/O, classes, and data structures is presumed. During this class, students will use some UNIX system calls to complete the assignments.

A significant portion of the class will involve the design and implementation of a major term project. The project will be developed by each student working alone. That project is the core of a new operating system implemented from scratch for an ARM processor. Initially, both the classroom lectures and the section meetings will be covering material important to the design and implementation of the final project. Later in the semester, advanced topics will be covered in class; however, both the class and sections will continue to support students as term projects progress. For the term project, students will continue working on and debugging their projects leading to their complete implementation and a final demonstration.

Because the course requires a significant term project involving both programming and hardware implementation, the assignments will be time-consuming; therefore, a significant time commitment to the course is necessary. Although the relevant experience of students in the class is usually quite diverse, depending on background, it is not unusual for students to spend 10-15 hours per week or more completing the readings and homework assignments. Although the computers are available more-or-less around the clock, occasionally they will suddenly become unavailable (this is known as a crash). As with all such events, they always seem to occur at the worst possible time. Plan your computer work so that it is complete in advance of the deadlines. You have now been forewarned!

Books/Course Bibliography:

All course books are available from the Harvard Coop.

Textbook:

Modern Operating Systems, Fourth Edition; Andrew S. Tanenbaum and Herbert Bos; Prentice-Hall, 2015, 2008; ISBN-10 0-13-359162-X; ISBN-13 978-0-13-359162-0

Optional UNIX and POSIX Library Function Reference Books:

Advanced Programming in the UNIX Environment, Third Edition; W. Richard Stevens and Stephen A. Rago; Addison Wesley Professional, 2013; ISBN-10 0-13-343547-4; ISBN-13 978-0-13-343547-4

PThreads Programming: A POSIX Standard for Better Multiprocessing; Bradford Nichols, Dick Buttlar, Jacqueline Proulx Farrell; O'Reilly & Associates, Inc., 1996; ISBN-10 1-56592-115-1; ISBN-13 978-1-56592-115-3

Recommended C Language Reference Manual:

C: A Reference Manual, Fifth Edition; Samuel P. Harbison and Guy L. Steele, Jr.; Prentice-Hall, 2002; ISBN-10 0-13-089592-X; ISBN-13 978-0-13-089592-9

There will also be other handouts & supplementary readings

Instructor:

Dr. James L. Frankel Dr. Frankel's Photo

Teaching Assistants:

We have two Teaching Assistants (TAs) for the course. The TAs hold a weekly section and office hours as described below. Attendance at the section is strongly recommended. When appropriate to send e-mail, please send e-mail to both TAs and to the course instructor.

TA Section Meeting Time/Place Office Hours Time/Place E-mail Address/Phone
Daniel Willenson
Daniel's Photo,
Section Site
Tuesday,
6:30-7:30 PM ET,
1 Story Street, Room 303
Monday,
7:30-8:30 PM ET by appt. only,
Science Center, Room 101e
E-mail: Daniel's e-mail address; +1.571.265.2932 (9:00 AM - 9:00 PM ET). If there's no answer, please leave a message with your name and a call-back number. Questions whose answers would be relevant to the whole class should be posed via Piazza. When e-mail is appropriate (for grading questions, personal issues, etc.), e-mail should be sent to both TAs and also to the professor.
Mark Ford
Mark's Photo,
Section Site
Tuesday,
6:30-7:30 PM ET,
1 Story Street, Room 303
Thursday,
6:30-7:30 PM ET by appt. only,
Science Center, Room 101e
E-mail: Mark's e-mail address; +1.978.496.7213 (1:00 PM - 9:00 PM ET). If there's no answer, please leave a message with your name and a call-back number. Questions whose answers would be relevant to the whole class should be posed via Piazza. When e-mail is appropriate (for grading questions, personal issues, etc.), e-mail should be sent to both TAs and also to the professor.

Students:

We'd all like to meet the other students in the class -- both near and far. Please use the Canvas Say Hello! facility to upload a brief video (about a minute or two in duration) introducing yourself to the class.

Grading:

Undergraduate-credit and graduate-credit students:

Problem Sets:

All problem sets and programming assignments are due at midnight Eastern Time on Sunday night (i.e., midnight between Sunday and Monday) unless otherwise stated in the assignment or in the syllabus. Unless otherwise stated, all programming assignment solutions must be written in C, have run successfully on one of the Linux/Ubuntu 64-bit x86 computers (collectively accessible via nice.fas.harvard.edu) in the Science Center or on the NXP/Freescale K70 Tower computer systems, as appropriate, be submitted using "git" on cr.cscie9x.net (or, in dire circumstances, via e-mail only if agreed to by your TA), be well-written (clear coding style, modular structure, appropriately commented and documented in English), and tested (include any programs and/or shell scripts used in testing your solution as part of your submission). Of course, the solutions may be written and tested using any system of the student's choosing; however, when the solution is complete, it must be pushed to the git code repository on cr.cscie9x.net. If you don't have your own native Unix/Linux system and run under Windows, you may find it useful to use code development tools under Cygwin. We will be grading the Unix/Linux solutions based on their behavior on the Harvard "nice" computers. Therefore, you should make sure that the final version of Unix/Linux programs that you submit for grading works properly on the "nice" computers. In addition, each submission must include a makefile to build the assignment. The grade for programming assignments will include all of these attributes. The Science Center computers may be accessed using "ssh" over the Internet. Files may be transferred to these systems using "secure ftp" (SFTP). The SecureCRT and SecureFX programs are available from the Science Center at www.fas.harvard.edu/computing/download; these programs implement "ssh" and "secure ftp," respectively. Remember, in addition to handing in all parts of the problem set solution or programming assignment program, sample runs of the program which demonstrate that the program works must be attached.

Some assignments may include Extra Credit problems. The Extra Credit problems can be completed to earn points that can increase the overall grade on your problem set; however, no complete problem set grade will ever exceed the full credit possible grade. That is, the Extra Credit problem(s) can be used to make up for deficiencies in other portions of the problem set to allow a higher grade to be earned. Extra Credit points from one problem set are not transferrable and may not be used on any other problem sets.

Late Policy:

A late homework will lose 5% of its original grade for each day it is late (e.g. an assignment handed in two and a half days late will receive its original grade multiplied by 0.85). Late assignments may be submitted via "git" and an e-mail message notifying the instructor and the teaching assistants should be sent immediately after the late assignment is submitted. In addition, each student is given five free late days that may be used freely during the semester. However, keep in mind that the assignments are often built on the previous assignments; handing in one assignment late does not extend the due date for subsequent assignments. The scope and difficultly level of the assignments increases during the class; therefore, we recommend against using the five free late days early in the class.

Commented and Documented:

In the "Grading: Problem Sets" section above, the phrase "commented and documented" is used; this paragraph will clarify the necessary comments and documentation that should be provided with all programs. First, there should be a description of the entire application. This should include the user interface (i.e., how a user interacts with the program) and an explanation of what the program does. This documentation may be in a separate file from the program itself. Second, there should be a description at the beginning of each file which outlines the contents of that file. Third, each routine, function, method, etc. must be preceded by a section describing: (1) the name of the routine, (2) the purpose/function of the routine, (3) the parameters to the routine (name, type, meaning), (4) the return value from the routine (type, meaning), and (5) any side-effects (including modifying global variables, performing I/O, modifying heap-based storage, etc.) that the routine may cause. Fourth, declarations of variables should be commented with their purpose. Fifth, blocks of code should be commented to describe the purpose of the code section. Sixth, any complex or difficult to understand code statements or fragments should be commented to clarify their behavior.

Programming Languages:

In addition to programming in a conventional language (the C Programming Language), students will learn how to write code in ARM/Thumb assembly language. These are the low-level languages used by ARM computers. All students are required to use Special Edition CodeWarrior for Microcontrollers, Version 10.6 or Kinetis Design Studio (KDS), Version 3.2.0 from NXP/Freescale. This software may be downloaded from the www.nxp.com web site, is free, and no license is required. The Special Edition CodeWarrior for Microcontrollers software runs only on Windows. The Kinetis Design Studio software runs on either Windows, Mac, or Linux (DEB or RPM).

Using git:

When using "git" and cr.cscie9x.net, make sure to follow the information on using "git" and setting up your repository that is available on the section web site. Create a named branch for each of your problem sets as follows: specify "problem-set-0" for Problem Set 0 (the course questionnaire & fix-this-program.c), specify "problem-set-1" for Problem Set 1, "problem-set-2" for Problem Set 2, etc., specify "project-proposal" for the Term Project Proposal, and specify "term-project" for the Term Project.

Midterm Exam:

Our midterm exam is a three hour long proctored exam. Students who live within the six-state New England area (Connecticut, Maine, Massachusetts, New Hampshire, Rhode Island, and Vermont) must come to campus to take the exam (of course, any student may choose to come to campus to take the exam). See Distance Learning Links: Midterm Exam for more information for distance students.

The exam allows open-book access to *only* the required textbook: Modern Operating Systems, Fourth Edition; Andrew S. Tanenbaum and Herbert Bos. No notes are allowed. No electronic devices are allowed.

Plagiarizing:

All work should be the personal creation of the individual student. Students are free to consult with each other and to study together, but all problem set solutions, programming assignments, exams, and the final project must be the personal contribution of each individual student. More explicitly, whenever a concept is reduced to a detailed algorithm or a program, no collaboration is allowed. If a paper, assignment, exam, program, or final project contains any information, algorithms, program fragments or other intellectual property taken from another source, that source and material must be explicitly identified and credit given. If you have any questions about this policy, it is the student's responsibility to clarify whether their activity is considered plagiarism.

Course Outline:

Approximate Schedule:

November 2016 Description
10 Online registration opens for degree candidates
14 Registration opens for all students

 

January 2017 Description
16 Martin Luther King Jr. Day
22 Registration ends
23 Classes begin
24 Introduction, course information & policies, outline, schedule. Devices. Abstractions provided by the operating system. Operating system structure.
30 Late registration (with $50 late fee) ends; last day for course and credit status changes; last day to drop courses with full-tuition refund.
29 at Midnight Problem Set 0 (the course questionnaire & fix-this-program.c) due.
31 Processes. Synchronization, scheduling.
For today, read Tanenbaum 4/e chapters 1 "Introduction" and 2 "Processes and Threads".

 

February 2017 Description
6 Last day to drop courses with half-tuition refund.
7 Finish covering processes. K70 hardware platform. CodeWarrior development environment. Basic Electronics. Projects DataSizes, FlashLED, and Pushbutton. Distribute hardware.
12 at Midnight Problem Set 1 due.
14 Finish covering projects FlashLED and Pushbutton. Memory management. Virtual memory, swapping, paging, and segmentation.
For today, read Tanenbaum 4/e chapter 3 "Memory Management".
20 President's Day
21 Input/Output systems, file systems, buffering. Numerical encodings.
For today, read Tanenbaum 4/e chapters 4 "File Systems" and 5 "Input/Output".
26 at Midnight Problem Set 2 due.
28 Finish covering Hamming Codes in Input/Output systems. Serial Communication. Project SerialIO. Introduce Deadlocks: definition, detection, recovery, avoidance, and prevention.
For today, read Tanenbaum 4/e chapter 6 "Deadlocks".

 

March 2017 Description
7 Application Notes. Deadlocks: definition, detection, recovery, avoidance, and prevention. Details of SerialIO project. Projects MCGInit, sdramTest, and LCDRGB.
For today, review Tanenbaum 4/e chapter 6 "Deadlocks".
12-18 Spring Break
19 at Midnight Problem Set 3 due.
21 Discuss UART2 problem on some Windows 10 computers and fix. Demonstrate UART2 use in SerialIO and in LCDRGB projects. Introduce concepts and code for using TWR-LCD-RGB, capacitive pads, A-to-D conversion, Supervisor calls, and privileged execution state.
28 Midterm exam.

 

April 2017 Description
2 at Midnight Problem Set 4 due.
4 Security and protection.
For today, read Tanenbaum 4/e chapter 9 "Security".
9 at Midnight Term Project Proposal due.
11 Introduction to parallel systems, data parallelism, communication, gang scheduling.
For today, review Tanenbaum 4/e chapter 8 "Multiple Processor Systems".
16 at Midnight Problem Set 5 due.
18 Synchronization, threads, RPC. System modeling, performance measurement and evaluation. Introduction to distributed systems, DCE/Encina. Distributed file systems. Fault tolerance, replication, redundancy. Case study of UNIX/Linux/Android.
For today, review Tanenbaum 4/e chapter 2 "Processes and Threads". For today, read Tanenbaum 4/e chapter 10 "Case Study 1: Unix, Linux, and Android".
21 Last day to withdraw from courses with no tuition refund. Course remains on record: final grade of WD assigned to credit students, final grade of WN assigned to noncredit students
25 Case study of Windows 8.
For today, read Tanenbaum 4/e chapter 11 "Case Study 2: Windows 8".
30 at Midnight Problem Set 6 due.

 

May 2017 Description
2 Operating system design, Virtualization, and additional topics.
For today, read Tanenbaum 4/e chapter 7 "Virtualization and the Cloud" and chapter 12 "Operating System Design".
8-13 Final Exam and Final Class period
9 Final Class Meeting during usual section and class time. Student project presentations/demonstrations.
12 by 2 PM ET Term Project report, slides, code, makefiles, test programs, etc. are due.
25 Commencement
29 Memorial Day

Hardware Related References:

The NXP/Freescale ARM:

This semester we will be using NXP/Freescale ARM development modules. The processor, serial, and connector boards are part TWR-K70F120M-KIT (which includes TWR-K70F120M, TWR-SER, TWR-ELEV). In addition, students will be using the TWR-LCD-RGB board. Students able to attend the course in person in Cambridge (on the day that we distribute the hardware and on the last day of class) will be able to borrow the hardware for use during the semester. All borrowed hardware must be returned at the final class meeting. Distance students must purchase their own NXP/Freescale TWR-K70F120M, TWR-SER, TWR-ELEV, and TWR-LCD-RGB boards. Additionally, some students may be able to use the following NXP/Freescale hardware: TWR-AUDIO-SGTL, TWR-ADCDAC-LTC, TWR-MC-LV3PH, TWR-RF-SNAP, TWR-SENSOR-PAK, TWR-SER2, TWR-WIFI-RS2101, and FSLBOT (includes TWR-MECH).

USB to Serial Adapter:

Segger J-Link Debug Probe:

System files for the Kinetis K70F120M/MK70FN1M0 (120 MHz), GCC Toolchain:

System files for the Kinetis K70F120M/MK70FN1M0 (120 MHz), NXP/Freescale Toolchain:

Sample programs for the Kinetis K70F120M/MK70FN1M0 (120 MHz):

CSCI E-92 Application Notes for using the Kinetis K70F120M/MK70FN1M0 (120 MHz):

The Renesas Technology M16C:

The Renesas SKP16C62P StarterKit Plus is an evaluation kit for the M30626 microcontroller. The M30626 microcontroller contains an M16C processor core with memory, peripherals, and an LCD controller. The Renesas SKPCOMMS Communications Applications card is a daughter board for SKP StarterKits that adds SRAM, two RS-232 ports, Ethernet, and two CAN ports.

Stack contents for hardware and software interrupt service routines on the M30626 when used via C

Memory map of the M30626 on the SKP16C62P StarterKit with the SKPCOMMS daughter board

All system files and sample programs below zipped together: M16Cfiles.zip

System files for the M30626:

Sample programs for the M30626:

The Microchip Technology PIC:

The example PIC assembly language program, blink.asm, from class to blink the LED's.

The computers for programming PIC's and testing PIC software are available in the PC Lab at 53a Church Street.

Other Components:

Electronics stores in the Greater Boston Area:

Electronics distributors on the Web:

Hardware (as in nuts and bolts, etc.) distibutors on the Web:

Static dissipative devices used in the lab:

The ring terminal at the end of the common point ground wire should be attached to the nearest electrical outlet faceplace using the screw between the outlets of a duplex outlet. Note that the alligator clips connected to the wires on both the anti-static mat and the wrist strap are removeable. The alligator clips are actually adapters from banana plugs to alligator clips similar to the Mueller Electric BU-60. After the alligator clip adapter is pulled off, the banana plug can be plugged into one of the two banana jacks on the common point ground.

Software Related References:

Look here for information about the GNU Project and the Free Software Foundation.

Look here for information about getting GNU Emacs for Windows 95/98/ME/NT/XP and 2000.

Look here for information about getting the Cygwin environment for Windows.

Look here for GNU manuals online.

Look here for Unix distributions and more.

Look here for information about getting mtools: A collection of utilities to access MS-DOS disks from Unix.

There are computers available for Extension student use in the Science Center and at 53a Church Street.

Harvard Arts and Sciences Computer Services (HASCS):

Tera Term Pro from UNC at Wilmington or from CNET is a free terminal emulator/telnet client for Windows. TTSSH adds SSH capabilities to Tera Term Pro with an open source/free software license.

Course Documents On-Line:

Slides used in class are available on-line:

The course questionnaire is available on-line. Please submit your completed questionnaire using "git" with the tag "problem-set-0" as soon as possible after our first class meeting.

The class problem sets are also available:
Problem Set 0 (the course questionnaire & fix-this-program.c),
Problem Set 1 (textbook problems & simple shell including conversion of microseconds since the Unix epoch to printable form),
Problem Set 2 (memory management: malloc & free, K70 LED flashing program, shell enhancements),
Problem Set 3 (textbook problems & device independent I/O for LEDs, pushbuttons, and an in-memory file system, shell enhancements, must use UART serial input & output for all shell interactions),
Problem Set 4 (K70 enhancements: faster clock, off-chip SDRAM, supervisor calls, serial I/O added to device independent I/O, also LCD display, A-to-D input, and touch sensors, unprivileged mode, shell enhancements),
Term Project Proposal,
Problem Set 5 (FlexTimer for time and date, interrupt-driven serial I/O, user timer, shell enhancements),
Problem Set 6 (SysTick for multiprocessing, processes).

A Piazza Wiki/Forum (on-line discussion list) for CSCI E-92 is set up at Harvard Extension School CSCI E-92 Piazza Forum.

Look here for information about the GNU Project and the Free Software Foundation.

Look here for information about getting GNU Emacs for Windows 95/98/ME/NT/XP and 2000.

Look here for information about getting the Cygwin Linux-like environment for Windows.

Look here for Unix distributions.

There are computers available for Extension student use in the Science Center and at 53a Church Street.

Harvard University Information Technology:

Here is an interesting discussion of the Mars Rover Pathfinder mission involving priority inversion.

Here's the discussion of the crash of the Ariane 5 rocket in HTML format and in text format.

For papers discussing the Internet Worm of November 1988, see note by Gene Spafford & Robert Gasch, General Accounting Office report, University of Lowell report, How to patch the holes, the court appeal, request for comments 1135, MIT-IEEE, MIT, seely, spaf CCR, spaf ESEC.

Description on blog.CryptographyEngineering.com by Matthew Green of the OpenSSL Heartbleed bug.

Description of Diffie-Hellman key exchange vulnerability by Alex Halderman and Nadia Heninger.

Description of Ingenious Chip-and-Pin Card Hack by Andy Greenberg of WIRED.

The C* Slides are available here in PDF format: The C* Language.

Section specific home page is available: