Blame

ba2ffc Dave LeBlanc 2026-10-27 06:34:12
History updated on Tue Oct 27 06:23:46 2026
1
# History
2
Originally beginning in the Computer Science Department, the systems administration team has evolved with the growing needs of the department, making policy and creating solutions to resolve an evolving need, as well as instituting policy to avoid potential future downfalls with regard to use cases that could possibly lead to Honor Code
3
mishaps.
4
5
Initially, the CS department was self-administered, with various postdocs, graduate students, and undergraduates disparately maintaining portions of the CS infrastructure. It was eventually realized that this methodology does not scale within a department, and stifles productivity in both systems administration and users.
6
7
In February of 2000, CS had the following issues:
8
- nothing is documented
9
- little or no discipline in setup (wiring, installation, versioning, documentation)
10
- passwd/hosts/fs/group setup is a non-standard, non-documented, complicated and error prone process
11
- machines are overloaded with multiple services causing interaction problems
12
- built on hodge-podge collection of equipment
13
- servers
14
- clients
15
- setup
16
- hodge-podge location and management of equipment
17
- (in the past) Too many bosses with own (not coherently interacting) agendas
18
- no stated policy on level of service target
19
20
The goals to solve these issues were presented (*circa FEB 2000*):
21
* Consolidate on a single, department-wide file server type.
22
* file servers *only* serve files.
23
* replace all existing file servers
24
* discipline ourselves not to just put disks anywhere
25
having them all be the same and only serve files probably gives us 80% of the benefits here; for the last 20%, we can haggle over the particular type.
26
* options on table:
27
* !NetApp
28
* lowest price runs $22k/100GB
29
* heard nothing but praise for these for uptime and reducing system administration time
30
* Sun RAID (viable if they would donate it to us)
31
* Linux PC+RAID
32
* was pricing $7K/100GB
33
* If use 50GB disks, could, perhaps, do $10K/200GB
34
* not locked into vendor, lowest price per GB
35
* *This only works if we can really do it department wide.*
36
* ITS runs network
37
* we can still run our DNS, allocate IP address in our range
38
* ITS manages wiring, hubs, etc. (labels them...)
39
* big benefit is it's simply not our problem
40
* Separate/dedicate servers for critical services.
41
* Service interactions are bad.
42
* forces compromises in how machines are run and managed
43
* increase likelihood of failure/downtime
44
* complicate rebuild/upgrade/migration
45
* Machines are cheap
46
* Servers identical generic setup, isolated uniqueness for service they provide.
47
* Central Services:
48
* DNS
49
* YP? / Hesiod? / whatever
50
* mail (? also need to separate imap and non-imap mailer?)
51
* web
52
* backup
53
* ftp
54
* license server(s) (maybe more than one)
55
56
* Straightforward account procedures
57
* Single canonical source for each dataset needed by machines/clients
58
* dataset
59
* host table
60
* passwd
61
* fs map
62
* groups
63
* printer info (printcap)
64
* documented addition procedures
65
* RCS or equivalent version logging
66
* Document procedures
67
* for users
68
* for professional or grad. student system hackers
69
* Clients"
70
* There are good reasons for wanting particular, perhaps diverging machines.
71
* At least, need to be identical within class.
72
* only differ in essentials (name, ip address, ...)
73
* centrally managed, standard application binary set
74
* Auto-reconcile with master
75
* group-specific software, live on group fileservers/systems
76
77
A Computer Science Department lunch time discussion took place, to answer lingering questions and inform faculty, postdocs, grads and staff about the plan to centralize and organize the hodge-podge that was the CS infrastructure at the time.
78
0ca19a Dave LeBlanc 2026-10-27 06:52:06
History updated on Tue Oct 27 06:51:31 2026
79
You can review the presentation in this [PDF file](Sysadmin/History/LB_oct17_2000.pdf).
e14cf1 Dave LeBlanc 2026-10-27 06:46:05
History updated on Tue Oct 27 06:43:48 2026
80
ba2ffc Dave LeBlanc 2026-10-27 06:34:12
History updated on Tue Oct 27 06:23:46 2026
81
82
Since that time, the CS Sysadmin Team has innovated services and infrastructure well beyond its beginnings in 2000, and was the first to implement at Caltech a long list of technologies, methodologies, and policies.
83
First implemented by CS at Caltech was:
84
85
* LDAP - Lightweight Directory Access Protocol, used in CS for authentication, authorization, mail aliases. Hardware and software inventory was stored in the LDAP directory for a short time as well.
86
* SNMP - Simple Network Monitoring Protocol, used in CS to actively and passively monitor resources to be aware of outages before CS users experienced a problem. SNMP in CS evolved into a trending and planning tool, to anticipate future needs for hardware to satisfy growing usage.
87
* SMTP-Auth - Simple Mail Authentication, used in CS to allow someone off-campus to route outgoing email through CS mail servers, effectively *relaying* mail but only after the sender has authenticated, to prevent unauthorized use.
88
* Maildir - an efficient mail storage mechanism in which each email message is a file on disk, implemented in CS to reduce or eliminate file-locking problems involved with traditional 'mbox' storage format in which all messages are stored in a single file.
89
* _access.conf_ - created as a Pluggable Authentication Module, by a CS postdoctoral scholar before he joined Caltech, to restrict remote access to specific users/groups, implemented in CS to deny mortal user access to infrastructure servers
90
* Documentation System - initially implemented for both system administrators and research members of CS, Computer Science had the first dynamic documentation system to catalog policies, FAQs, procedures for new users. It has evolved through several iterations over the years, beginning with Zope, then a Content Management System (CMS), then ikiwiki, a wiki-compiler in which documents are stored in a version control system, then [FosWiki](http://foswiki.org), finally this wiki.
91
* Canonical configuration management system - originally utilized to handle the automated configuration of a few dozen !FreeBSD hosts in the Intel Lab at the time, this has also evolved into a bi-directional automated system in which changes are reported to a central mechanism to confirm configuration changes
92
93
Typically, these innovations were accomplished out of pure necessity to provide a scalable sustainable environment with limited manpower.
94
Since accomplishing this goal, the systems administration team has progressed to anticipating user requests within the department in order to better serve an ever-changing need.
e14cf1 Dave LeBlanc 2026-10-27 06:46:05
History updated on Tue Oct 27 06:43:48 2026
95
f8d517 Dave LeBlanc 2026-10-27 06:53:33
History updated on Tue Oct 27 06:53:25 2026
96
{{AttachmentList|format=details}}