Monday, August 7, 2006

Dell 2007FP and xorg.conf

In order to better serve those folks Googling "2007FP xorg.conf", I've included an annotated xorg.conf file that seems to work well for me and my Dell 2007FP monitor. I hope it helps you. (It is, naturally, provided "as is", without warranty of any kind, etc., etc., because it could very well damage your hardware, although nowadays monitors are much more forgiving than they once were.)

(It seems from my site logs that my venting about my adventures with Linux, xorg.conf, and the 2007FP is one of the more popular Google hits for my site now, just one day after it was posted.)

Wednesday, August 2, 2006

Venting about Linux

Time for a "Linux is not ready for the desktop" rant.

Some time ago, I picked up a Slackware 10.1 4-disc set. I've been trying to use it on a Dell Precision 610 MT workstation, Xeon Pentium III. (That's what you can read on the label.) Through some struggling through the initial start up screens, I was able to determine the box has two SCSI internal drives. I also have a Dell 2007FP monitor (pretty nice).

Getting X to come up in something other than 1024x768 was an ordeal. Perhaps I should have realized that no amount of so-called "handholding" was going to help me, and gone to the xorg.conf man page directly. Silly me, I thought this would be like a normal desktop environment, where I would open something called something like a "control panel" and find a sub-thingy labelled "Display" (KDE buries this under Peripheral) and select the 1600x1200 resolution my display claims to support. No. Only 1024x768 or worse.

So I go to; the FAQ is skeletally brief. It talks about xorgsetup (must run as root, not just using "su" but "su -"), which dumps a uselessly generic xorg.conf file in /etc/X11. xorgconfig asks me to answer vague questions about my mouse and keyboard EVERY time, and scroll through long lists of video cards, none of which is a close match to what the Dell startup screeen says: "Diamond Viper V770D 32MB" It asks me for refresh rates, but doesn't allow me to specify that "at 1600x1200, this monitor only allows 60 Hz vertical refresh". The slackware page config/x.php says something vague about "run X -probeonly" but "startx" seems only to feed its arguments to xterm, then puke when xterm does not accept it. I cycle through various efforts, copying the result to /etc/X11/xorg.conf, and get a blank screen with the monitor complaining "frequency out of range: use 1600x1200@60Hz."

A search through the the Video HOWTO for Diamond Viper 770 shows me RIVATNT2 XF86_SVGA and nv.

Finally, I break down and edit xorg.conf by hand. It takes a while but I start to realize that various identifiers (described just as "free form" by xorgconf, actually are used to refer to various entities in later descriptions. I hand edit my xorg.conf to indicate my "Monitor" can HorizSync at 30-82, only VertRefresh at 60 Hz, my "Device" has driver "nv", and my "Screen" is that Mointor, Device, with Modes "1600x1200" "1280x1024", etc.

Finally. It seems to work.

Next up, try to get USB flash memory key to be recognized, without following directions which tell me to use the SCSI /dev/sda files that are, ahem, being used for my SCSI hard disks.

[UPDATE: Fixed the link to the Linux Hardware-HOWTO video section.]

Tuesday, August 1, 2006

Clive Page’s Free Fortran 77 guide

Clive G. Page's Professional Programmer's Guide to Fortran 77 is available under the Gnu Free Documentation License. I suppose this means I should dig up my texinfo translation of his LaTeX, which I did a number of years ago, merge in his more recent changes, and release it.

Monday, July 31, 2006

More adventures in multithreading

Just wanted to capture a few more thoughts; over the weekend, I experienced what I think was my first case of deadlock. My chaosnet-on-OpenMCL implementation had worked just fine when the two sides of a connection were careful to exchange only one packet at a time, taking turns. When I tried to listen to a SEND (the Lisp machine analogue to an instant message), the CADR emulator sent two packets in quick succession, and my OpenMCL process would freeze trying to receive it.

As an aside, I get the feeling that I don't really know how to debug this kind of thing in OpenMCL: looking at backtraces does not always make it completely obvious which semaphore or lock is being waited for in the various threads, and I don't think it is possible to look at semaphores to see which thread(s) are waiting.

In any case, I concluded that my somewhat haphazard use of a "count lock" to protect access to packet number information in a connection was either being used recursively, or violating a lock heirarchy. So, after some time trying to apply my powers of deduction, I decided to look at the ChaosNET problem from a high level to determine what locking is actually necessary.

The main complexity has to do with the transmission window. In theory, it seems like an ideal case for a counting semaphore: the send window is a limited resource, and sending processes must wait until a spot within the window is available. (Generally, one would wish to preserve FIFO order, but strictly speaking, multiple threads trying to transmit on the same channel might not deserve any guarantees that their packets will happen in any particular order.)

However, there are several things unusual about the window: the thread handling incoming packets can get both acknowledgements in ordinary packets, as the receiving user process consumes packets from the window, and in STS packets that could, at any time resize the window. In the usual semaphore model, it is not easy to deterministically reduce the maximum level of the semaphore. I coded an unsatisfactory approach which does a trial-wait-on-semaphore for the number of window packets that have been taken away. But these might not succeed, if the window is filled above the new capacity. In that case, the window must presumably be allowed to drain below the new level before any waiting process gets allowed to put packets in. It might be possible to have a separate integer variable tracking the "excess window consumption," and for processes to loop back and wait again for the semaphore if the window has been "tightened" since they first waited. I don't know if it is possible for threads to maintain their original FIFO order in this case, or whether it matters. (I'm not sure it is reasonable for Chaos processes to reduce the window, once it is declared in the initial STS negotiating the connection, as opposed to widening it to allow for better throughput. The AIM memo is silent on this.)

Interestingly, the draft chaosnet memo does discuss enlarging and shrinking the window, and mentions WIN and WTS packets.

In terms of avoiding deadlock, there are many participants: the user reading process looks at the window to see if the connection might be getting "stymied", and needs an STS to be transmitted. When STS or acknowledgements come in, the retransmission of the receipted or acknowledged packets should be suppressed, and the acknowledgements (but not receipts) open up space in the window. In the Lisp Machine design, the receive-sts et al. use the RECEIPT function to trim the send-pkts list. I felt that contending with the sending/retransmitting process was asking for trouble. Therefore, I chose to have the retransmission process look at the receipt and acknowledgement status to clean up the send-pkts while it needs exclusive access to the send-pkts list anyway. The retransmission process needs access to the send-pkts list to retransmit, and to the counters if it is required to do clean up. Furthermore, updates of the acknowledgement and receipt states should rachet only forward, i.e., a stale packet coming through the pipe should not erase the acknowledgement or receipting indicated by the logically latest packet received up to this point. That requires locking to protect the read-modify-write update of those status numbers.

One interesting gap in the ChaosNET documentation is whether retransmitted packets can or should update their acknowledgement and receipt information to reflect what has happened to the connection in the meantime. The Lisp Machine implementation seems to retransmit the packet byte-for-byte. An alternative would be to update those packets. In practice, I suspect retransmission should be conservative; it is necessary when the receiving process has fallen behind, or the channel is experiencing loss, and the bias should be toward more retransmission than strictly necessary, instead of hoping to "make up for lost time" by advancing the state of the return channel.

I have created enough of a system to work with the SEND transaction, except for one detail: in a Chaos "stream", my functions indicate an EOF condition only when the request for a byte has gone "off the end." I felt the EOF detection should predict whether the next byte request should succeed; but I could not remember how this works elsewhere.

Tuesday, July 25, 2006

Apple II Disk Transfer

It seems that others have also had the desire to upgrade the ADT (Apple Disk Transfer) tool to work with disks other than the Disk II 5-1/4 inch floppy format under DOS 3.3.

ADTPro is a ProDOS-based version of ADT. It should presumably work with all block devices recognized by ProDOS. I e-mailed some code to David Schmidt which allowed ADT to work with the ordinary IIgs serial ports; direct access to the Z8530 chip might allow for faster transfers; that's a potential project.

Friday, July 21, 2006

Clickr: Lisp API for Flickr

Mark Probst has developed a Common Lisp API for Flickr and mentioned it on his blog.

Wednesday, July 19, 2006

More thinking about multithreaded queueing

Looking back on my previous post about multithreaded queues, and based on a little hacking I was able to do last night on my multithreaded-Lisp-user-mode-Chaos-client, and having read a bit more in the Little Book of Semaphores, (must stop typing 'sempahores'!), in particular the turnstile pattern, and the barbershop problem, there are a few other ways to slice the onion.

  1. A "waiting room", i.e., the privilege of blocking on the queue is exclusive. That actually makes a certain amount of sense when, for instance, Chaos packets are already sorted by the destination connection. Does it really make sense for multiple consumers to be concurrently transacting on a single connection? What would make sense, in a C10k kind of way, is rapidly generating server connections in response to RFCs, and handing these connections to waiting server threads. Or, if a connection is known to have independent incoming packets, or is deliberately recycled, connections can queue through a turnstile, each take one incoming packet, then go off to chew on it. I think the mechanism could be a simple lock.

  2. "Signal enough" can be expanded to include a "turnstile" approach, where a disappointed consumer signals the semaphore on his way "out" so that the next waiting consumer can be disappointed as well. The idea that the reason for the disappointment should be visible to incoming consumers before they block is really optional. The main reason to prefer it is that, if we believe the "disappointment" is permanent, that we will eventually want to discard the queue, but we presumably cannot do so if consumers can continue to block. That is, if the disposal process must race against the arrival of new consumers. Discarding a queue when the semaphore still indicates the presence of (non-existent) packets and no threads are waiting is OK. Discarding a queue when the semaphore indicates no packets, but threads are blocking requires a well-defined mechanism for blocking threads to cleanly deal with a "killed" semaphore. I.e. the "abort" mechanism.

I really like the Little Book of Semaphores. However, I find it hard to be sure that I truly grasp the concepts. I can read through the solutions to the problems, and understand the discussion, but that doesn't give me a huge amount of confidence in my own solutions to slight variations on the problem. Maybe this kind of programming is simply hard.

I've tried casting some of the solutions in OpenMCL Common Lisp. It's hard to verify that things don't work by accident. One thing that wasn't particularly clear to me is whether it is kosher to do things like

(let ((semaphore (ccl:make-semaphore)))
(labels ((client-function () #|...|#)
(server-function () #|...|#))
(let ((thread1 (ccl:process-run-function #'client-function))
(thread2 (ccl:process-run-function #'server-function)))
(#|...some way to wait for both threads to terminate..|#))))

It seems to work, except I haven't yet tried to make a clean "wait for both threads to terminate", and have just waited for the length of a global list to get to a certain point, or similar kludges. The idea, however, that the let can create "global" variables, but the lambdas can have "thread-local" storage, seems not to be formally specified.

Wednesday, June 28, 2006

Some old thoughts about TeX

I saw a link on reddit the other day regarding some math-related videos, including a series of lectures by Don Knuth, including some on TeX internals. (One frustrating bit about the Web video is the blurriness of his terminal. Another is that he uses what is now a slightly-out-of-date version of TeX itself).

He made a comment near the beginning of section 3, probably similar to things he has written, to the effect that in his education he had benefited both from reading badly-written programs, because they were evidence he could do better, and from well-written programs, because they were a pleasure to read. He then wondered aloud which category TeX is in, but that either way it was a win. I'm not at all sure myself. Perhaps TeX is a monument to the best that can be done in Pascal, and simultaneously a warning that it is the best that can be done in Pascal.

I had a definite feeling during grad school, when I was learning LaTeX to write my thesis, and learning Lisp on the side, that a large chunk of TeX internals was spent managing things that manage themselves in Lisp
  1. Dynamic memory (including variable-length strings and reference-counted objects)
  2. Interned static strings (i.e. symbols)
  3. Rational arithmetic protected from overflow and portable between machines
  4. Heterogeneous lists

Knuth also spent a good deal of time, however, fretting about the "inner loop" of TeX, so perhaps re-implementing it in a higher-level language would incur a performance penalty of large constant factors in the runtime of TeX.

I spent a few odd hours with TeX, The Program, coding a few bits in Lisp, as well as beginning a few sketches that could serve as commentary. (One of the most grievous flaws in TeX, The Program is its utter lack of diagrams, apart from the rough memory map on the back flyleaf.) The main topic of the commentary was to sketch memory structures and the overall memory map in diagram form.

Another grievous flaw I saw, perhaps related to the efficiency concerns, was the monolithic architecture. He speaks of TeX's eyes and mouth and digestive tract, but the WEB presentation blurs this all together; as far as I could tell with my limited reading the TeX syntax is processed directly into lower level memory structures without any clear documentation of how those memory structures should be thought of as logical constructs. That is, there is no higher order description of how TeX should behave in the code, simply procedures to take TeX and produce almost raw bytes, and then slurp up those raw bytes into DVI output, with certain very carefully designed algorithms for math layout and paragraph breaking hidden inside.

Knuth's approach to the lack of abstraction in the Pascal expression is to add English prose, both in the WEB format of the code, and in the TeXbook's user documentation, but not to present a more abstract, formal, or diagrammatic description of the architecture. (He does describe some small algorithms in more precise ways, such as the DVI optimizations using down_ptr and right_ptr.) I would hope to replace this by an elegant Lisp description of the underlying structures and algorithms, leaving the messy hand-made arithmetic and memory management off to the side as part of the unfortunate need to run on 1982-era platforms. But I get the nagging feeling that the whole design of TeX the language and the data structures and the algorithms is just one big ball of mud that will resist such analysis.

I am slightly annoyed by the use of numerical coincidence (e.g. using the numerical ordering and grouping of an enumeration to simplify some decision logic) and a few related hidden limits (must get my copy of TeX, The Program to see where I wrote one in the margin). Seems rather too low-level for someone trying to present things clearly.

Another one of my past TeX-related projects was to try to get a from-scratch Pascal-based WEB system producing TeX on Mac OS X, so that I could really understand the WEB code, e.g. by enhancing the TeX internal debugger, instead of simply using a Web2c translation.

Sunday, June 4, 2006

Lisp hacking

I wanted to store a link to Alastair Bridgewater's (a.k.a. nyef on IRC) SBCL-based LispOS notes.

Nyef's most apparent distinguishing characteristic is an amazing willingness to hack low-level internals on Intel x86-based Lisp implementations. Given the claim that "The classic problem with the 'LispOS Project' is that a large portion of the early effort would involve low level hardware hacking", nyef seems to be the ideal candidate for overcoming that problem.

Also, just wanted to record a link to a vendor selling the remnants of the Interlisp environment. Seems like for something under US$3000, one can purchase a x86-Linux-compatible version of the Interlisp programming environment.

Thursday, February 23, 2006

link dump for 22 Feb 2006

A bunch of stuff I was reading back then

Weak Data Structures write-up by Bruno Haible, courtesy of Gary King.

The classic UNIX HATERS Handbook.

Panda's TOPS-20 System.

Spare Time Gizmos store front to buy actual hardware emulators of the PDP-8.
Rainer Joswig e-mail on feel of old Symbolics Lisp Machines

An e-mail discussion about CHAOS vs. TCP in port collisions, a list of Symbolics CHAOS services

Google cache of mahalito
Symbolics Y2K statement
ITS site HACK directory

Google Groups discussion on Basic Chaosnet information
Google Groups discussion on Chaosnet and ARP
Google Groups discussion on Transmitting Chaosnet over Ethernet
Google Groups discussion on Why Chaosnet (nostalgia)
Google groups search for chaosnet

ITS build information build.doc.txt
Retro PDP-10

Thursday, February 16, 2006

Some ITS links

Wanted to collect a few links on ITS, the Incompatible Time-Sharing System

Tuesday, February 14, 2006

Some ChaosNET thoughts

I looked a little more closely at Bjorn Victor's (TODO: add diacritic to o in Bjorn) work on Chaos emulation for ITS emulators, particularly the CHUDP protocol for passing Chaos packets through UDP.

Apparently, it uses a simple header (different from the user-mode UNIX header), and dynamically adds to its routing table when it receives incoming packets. This seems particularly handy.

Also, I noticed that the chaosd/server code for the user-mode UNIX implementation does not typically check the hardware destination, but jumps right to the software header for the destination information. This seems wrong.

I'm trying to figure out the most useful way to improve the Chaos support. Some of the use cases would be

  • use Internet hosts as repositories for CADR Lisp Machine microcode and "world" bands

  • use Internet hosts as file servers for Lisp machine source code

  • provide Internet-based early-21st-century substitutes for 1980's functionality, such as e-mail and instant messaging (e.g. get Zmail to use Google's Gmail service, use Jabber or AIM to support instant SEND messages, use NTP to provide TIME service)

Some of these would simply be a local process posing as a Chaos server which could respond to various contact names with useful functionality, configured by various users to, e.g., use the instant messaging protocol or e-mail provider of choice. Some of this would require or allow collaboration between nostalgists: someone volunteers, similar to the ITS community, to host the equivalent of AI, and provide ITS-like FILE service on an IP address & port, with a corresponding Chaos subnet/host address.

I suppose part of my problem is I don't understand all the issues involved in internetworking. Chaos addresses were supposed to be roughly "institution-wide" with specific gateways to other networks, advertised by response to a particular Chaos contact name (such as ARPA). How does this translate to isolated hosts on the Internet posing as institutions with Chaos servers? Can we replace it with a DNS-mediated free-for-all where various IP-based users advertise Chaos DNS entries? How would subnets be allocated? Or will we depend on strictly local, handmade routing tables, where each chaos address gets mapped to an IP host & port? In this case, how do the Chaos source and destination addresses get fixed up when they make it to the IP host & port? Can the sender and receiver basically hide their respective internal Chaos subnetting schemes from each other?

Thursday, February 9, 2006

What’s in a name?

I started this blog with a perfectly cryptic name: "wpblog." I suppose it is a natural impulse shared by many people who download WordPress and are faced with a blank box. I spent an additional minute on the issue today, and decided this was a blog about "getting down to brass tacks." Hence, my current name "BrassTacks." I'm not sure how good Google is at measuring the "uniqueness" of this sort of thing. "Voltaire" as the web host was a nod to my previous blog impulse left unfulfilled at, when I was hoping to develop a vicious talent for satire. The name alone proved ineffective, but easy to remember.

It turns out I have a few good names stored up; one is a nom de guerre, which, for security reasons, I will keep secret. I also have here a good name for an instrumental group: "The Cooper Brass Choir." Properly euphonious. If you have a good enough name for a techie blog, you may have it in exchange.

Sunday, January 29, 2006

What is this about

This blog is a simple experiment for now, to understand what is available for a "scratchpad on the Web."

I have a picture which is something more like a lab notebook for my various hacking projects. The requirements are roughly
  • access from various places where hacking might happen; an idea for a project can happen at work, but I don't want my personal laptop with me at all times.
  • some kind of permanence, as suits a lab notebook. Various things this might mean:
    • version control with timestamping
    • no revisions allowed at all (this seems extreme: one wants to separate a polished presentation view from the underlying archival lab notebook. One can revise slides for a talk without altering the historical record of the lab notebook)
    • ability to archive into a big .tgz for burning to disk as necessary
  • flexible project documentation
    • easy way to archive associated files; diffs, error messages, screen shots, sketches, links, data sheets downloaded for the web.
    • perhaps I need to wait for my copy of Kanare's Writing the Laboratory Notebook (as seen on Lispmeister) to arrive to understand what should be required here.

Some projects to comment on here
  • work on CADR emulator and MIT-released Lisp machine code
    • make it work on Mac OS X/PPC for my laptop
    • make Chaos emulation work on Mac OS X
    • fix some Y2K-type issues, site configuration
    • create Chaos layer in OpenMCL or more portable Common Lisp, to develop gateways to things like
      • AIM/Jabber
      • regular e-mail
      • other retro people running Chaos emulation
      • support versioning file system semantics (e.g. blah.type~version~) on a UNIX FILE host.
      • build a world from the MIT-provided code, alllowing bootstrap
  • Apple II file transfer (extend ADT = Apple Disk Transfer) to include 140k, 800k ProDOS and Macintosh HFS floppy formats
  • Possibly, an Apple II compatible USB peripheral interface, to allow easy transfers to a Mac/PC with USB connection
  • Mac OS X support for foot-pedal shift & mouse-click operations

Interesting that [for Wordpress] Netscape 7 on Windows gives me little editing widgets on my post that Safari on Mac OS 10.4 didn't give me (as I recall.) Perhaps I should check, try to add that capability.