Brasstacks

Restoring the Heathkit Jr 35

2011-12-23T11:30:00.000-08:00

Heathkit Jr Electronic Workshop "35", Model JK-18

The Heathkit Jr 35 evidently had been stored with batteries in the battery holder, which corroded badly. I replaced it with a plastic RadioShack 4 D-cell battery holder (270-389), wired into the negative power rail and the hot side of the power switch (which is part of the variable resistor).


Corroded battery holder

Replacement battery holder mounted

I mounted the replacement battery holder with a dozen "heavy duty" 1 inch foam double-stick mounting squares (rated to hold 900g), stacked in four groups each three high to overcome the various screws protruding through the main board.

The remote speaker station

The rubber foot used to protect the relay from being crushed when you turn the kit over was stuck to the remote speaker station. I put it in its rightful place.

Relay with the brown rubber foot

Earlier, I had built a couple simple circuits, which showed the meter and new battery pack working, but showed me the telegraph key switches were not making good contact. I used a Scotch-Brite scrubbing pad to remove corrosion from the key switches on the main board and the remote station.

I ran through the tests in the manual appendix to check out the lamp, power switch, speaker, earphone, antenna coil continuity, remote station speaker & telegraph key, slide switch, and relay.

I then jumped ahead of myself, building the 4-transistor AM radio experiment; I got slight hints of a audio signal if I wiggled and touched some of the wires. I'm highly suspicous of the electrolytic capacitors. I probably should have tested those (and the transistors) before using them in such a complex circuit. The wire connections to the springs are not all that reliable, and the wires might have some oxidation on them.

Heathkit Jr. 35

2011-10-01T04:23:00.000-07:00

I just picked up a Heathkit Jr. "35" circuit experimenter kit on eBay. I had one of these as a child; I don't remember how it is that we came to own it, but I had an itch to re-read the manual. Buying the manual from vintagemanuals.com would have cost as much as the kit + manual. I remember being befuddled by the descriptions of how transistors worked. It turns out the descriptions of capacitors and transistors were a bit sketchy: I think it is pretty much impossible to understand how the AM radio experiments actually function from the text. The operation of the ferrite-core antenna and the various windings are unexplained as well.

I was a bit surprised to find all four transistors in the kit are identical. Thanks to this Heathkit part cross-reference, I see that they are part 417-118, a.k.a. 2N3393 NPN transistors.

Importing from Wordpress.com

2011-09-24T10:25:00.000-07:00

I'm working on importing my blog from Wordpress.com. Blogger.com apparently will let me use AdSense even with free hosting, and I should get to know more about AdSense and about the Blogger platform as well, as a Googler.

I used the Wordpress.com export feature to create an XML file, and used the http://wordpress2blogger.appspot.com/ tool to convert it, as I was under the 1 MB size limit. The code behind that converter is evidently at http://code.google.com/p/google-blog-converters-appengine/.

That tool plus Safari saved the result with an .xml.html extension, which resulted in a Blogger error code of "bX-uroqxm" when importing. Removing the extraneous .html allowed the Blogger import to mostly succeed.

It does not re-map my Wordpress author identity to my Blogger profile. It also does a pretty poor job importing my source-code formatted content. So I'll be slowly publishing the old posts, tweaking that kind of thing as I go.

ChaosNET implementations for PDP-10

2010-02-10T15:01:00.000-08:00

This is not particularly interesting, but I wanted to record a couple of links to ChaosNET code for the PDP-10. The ChaosNET memo describes an ITS (Incompatible Time-Sharing System) interface, for which the code is somewhere in the ITS sources, and a Lisp Machine implementation, for which code is in the CADR sources and Symbolics code base, but also a TOPS-20/TENEX implementation, a VMS/VAX implementation, and a UNIX implementation. The UNIX implementation was documented by James E. Kulp for Symbolics, and the code seems to have survived. The TOPS-20/TENEX is harder to find, but it looks like BitSavers' Foonly archives have some files.

DSK:<FOONEX>CHAOS.MAC;399
DSK:<FOONEX>CHABUG.MAC;8
DSK:<FOONEX>CHAOS0.MAC;3
DSK:<FOONEX>CHATTY.MAC;21

I have no idea whether this is at all related to the implementation the AI MEMO mentions.

A Google search turns up some interesting code (might be for the PDP-11 front end, for ITS)

from AI:SYSTEM

CH10.defs1 "PDP-10 chaosnet interface definitions"
CH11.defs1 "CH11 Unibus chaosnet interface definitions"
CHSDEF.15 "Chaos net packet definition (for PDP-10s)"
CHAOS.288
KSNET.1 "Chaosnet interface on Unibus" (presumably for the KS10)
IOELEV.432 "i/o eleven"
NET.32 General ITS networking code

Another Symbolics keyboard adapter

2009-05-24T09:59:00.000-07:00

Hans Hübner has been at it again, and has developed code to allow a Teensy AVR development board to adapt a Symbolics keyboard to USB.

I ordered a Teensy board of my own, and was hoping to develop a small adapter to avoid cutting up my keyboard cable, but Teensy does not work with my old Mac OS X 10.4 system.

News on Symbolics keyboard adapters

2008-06-09T16:37:00.000-07:00

I wanted to post a link referring to recent progress on adapting Symbolics keyboards to modern PCs. Hans Hübner announced that the kbdbabel project (apparently based on 8051 microcontrollers) was able to be adapted to support the slim-line (Rev. C) Symbolics keyboard, adapting it to PS/2 protocol.

My own work in this area ground to a halt pretty soon after I received the ADB adapter from DKS. I'm happy to see that others have been able to make progress!

UPDATE: Rainer Joswig has commented on his experience using the kbdbabel adapter to connect a Symbolics keyboard to a Macbook running Mac OS X.

UPDATE 2: Hans has developed a USB adapter solution.

COMFY-6502: hosting at github

2008-05-06T02:50:00.000-07:00

Mostly to support my own development on multiple computers, I've moved my Common Lisp port of COMFY-6502 to github. I chose the name cl-comfy-6502 for the repository, although the "cl" prefix is a bit ugly.

The current snapshot of COMFY-6502.

Monthly Boston Lisp Meeting: report

2008-03-05T01:46:00.000-08:00

Thanks to François-René Rideau (fare) for organizing last night's inaugural Monthly Boston Lisp Meeting (Note: future meetings will apparently happen on the fourth Monday of the month.)

I'm surprised I haven't seen reports on Planet Lisp yet, but I had an enjoyable time. I certainly didn't meet everyone there, but did get to meet nyef (who showed off some of his fearless adventuring into bare-metal Lisp) and Brad Parker, got to see Kent Pitman in person, and a few other notables, as well as numerous ITA Lispers. Discussion was wide-ranging (I heard only a fraction of the multiple conversations going on), and some of the most interesting stuff to me was discussion about the background of Dylan and Lispers at Apple in the 1990s.

There was at least one person taking pictures; I'm sure they'll pop up somewhere in the blogosphere.

Looking forward to more great meetings in the future!

COMFY-6502: snapshot

2008-03-05T01:34:00.000-08:00

Even though it is hardly a finished product, I've posted a snapshot of my current CL conversion of COMFY-6502 Not that I expect a huge pent-up demand for the code, but it might be slightly more interesting than the output examples. Thanks to Henry Baker for allowing the release.

The page also contains a link to the TODO file describing my next steps. (Most immediately, my attempt to include JMP elision was messy enough to convince me I need a different class to represent opcodes+arguments as a unit.)

COMFY-6502: a slight correction

2008-02-11T01:57:00.000-08:00

After all the work to write a post showing off COMFY-6502's ability to reproduce the Red Book tone routine, I made a slight mistake in translation. The code I presented differed ever-so-slightly from the Red Book routine in the case where the Y-register reached zero.

In the original, when the Y-register reaches zero, the DEX is still executed. In the code I posted, the expiring "duration tick" skips a count on the pitch. A minor error, but one I should own up to.

Here's a better effort. COMPILE-SYMBOLIC contains a MACROLET defining MODULE as a call to COMFY-6502:COMPILE.


(compile-symbolic
 (equ speaker #xc030)
 (equ duration 1)
 (equ pitch 0)
 (module
  (comfy-6502:loop
     ;; repeat until some clause loses: actually, only exit is
     ;; through return, so each clause should be ensured of winning.
   (LDX pitch)
   (LDA speaker)
   (not
    (comfy-6502:loop  ;; repeat until whap time
    DEY
    (not (seq zero? (DEC duration) zero? return))
    (seq DEX (not zero?)))))))

resulting in


(; reload-pitch
(LDX :ZERO-PAGE) (:ZERO-PAGE 0)
(LDA :ABSOLUTE) (:ABSOLUTE 49200)
; spin
(DEY)
(BNE) (:BRANCH 6) ; not-duration-tick
(DEC :ZERO-PAGE) (:ZERO-PAGE 1)
(BNE) (:BRANCH 2)
(RTS)
; not-duration-tick
(DEX)
(BEQ) (:BRANCH 4) ; goto-reload-pitch
(JMP :ABSOLUTE) (:LONG-BRANCH -12) ; spin
; goto-reload-pitch
(JMP :ABSOLUTE) (:LONG-BRANCH -20) ; reload-pitch
)

For fun, I tried to get the code to emit the RTS at the very end, as in the original.


(compile-symbolic
 (equ speaker #xc030)
 (equ duration 1)
 (equ pitch 0)
 (module
   (alt (loop (seq (LDX pitch) (LDA speaker))
           (not (while
                   (seq DEY (not (seq zero? (seq (DEC duration) zero?)))
                   (seq DEX (not zero?)))))
         RTS)))


(; reload
 (LDX :ZERO-PAGE) (:ZERO-PAGE 0)
 (LDA :ABSOLUTE) (:ABSOLUTE 49200)
 ; spin
 (DEY)
 (BNE) (:BRANCH 5) ; not-duration
 ; duration-tick
 (DEC :ZERO-PAGE) (:ZERO-PAGE 1)
 (BEQ) (:BRANCH 10) ; exit
 ; not-duration
 (DEX)
 (BEQ) (:BRANCH 4) ; goto-reload
 ; goto-spin
 (JMP :ABSOLUTE) (:LONG-BRANCH -11)
 ; goto-reload
 (JMP :ABSOLUTE) (:LONG-BRANCH -19) ; reload
; exit
 (RTS))

"Loop tightening" could change (BEQ) (:BRANCH 4) to (BEQ) (:BRANCH -14); the (JMP :ABSOLUTE) (:LONG-BRANCH -19) is then never used, and could be omitted, changing (BEQ) (:BRANCH 10) to (BEQ) (:BRANCH 7).

"Loop compaction" could potentially recognize (BEQ) (:BRANCH -14) (JMP :ABSOLUTE) (:LONG-BRANCH -11), where no other instruction references the JMP, could be converted to (BEQ) (:BRANCH -14) (BNE) (:BRANCH -11), shortening the (BEQ) (:BRANCH 7) one byte more to (BNE) (:BRANCH 6).

Note some interesting equivalents I should probably implement as COMFY macros.

(if A B C) where C is do-nothing-but-win is (not (seq A (not B)). If A wins, B is executed to determine the result of the form. If A loses, the if wins. I will probably call this (when A B).

(if A B C) where B is a do-nothing-but-win is (alt A C), where alt is the "dual" of seq. (alt A B C ...) == (not (seq (not A) (not B) (not C) ...). For the two-argument case, I would call this (unless A B).

The one twist I can think of is to add an "implied SEQ" to these forms. (when A B C ...) would be (when A (seq B C ...)). (unless A B C ...) would be (unless A (seq B C ...)).

COMFY-6502: work in progress

2008-02-05T00:49:00.000-08:00

As I've mentioned before, I've become intrigued by Baker's COMFY assembler, and have been working on porting it to Common Lisp, and making it a bit more powerful in the link stage.

One metric to judge the success of this kind of "medium-level" language is how well it compiles compared to hand-written assembler code. For the 6502, there are a few examples of code created by wizards like Steve Wozniak, which you can find copies of around the web, the largest being the Apple II monitor, the Integer Basic interpreter, and some medium-sized ones like the Apple II 6502 step/trace, the mini-assembler, the floating-point "Wozpack," and the Sweet-16 virtual machine.

This kind of code has a lot of quirks that make it hard to straight-forwardly translate: lots of shared "tail code", branches known by the programmer to always be taken (to save a precious extra byte consumed by an unconditional JMP), and the classic "fake RTS trick", pushing a return address picked from a table onto the stack, then using an RTS instead of a zero-page indirect JMP. Common in Woz's code is a further shortening of the code by arranging the destination addresses to all be in the same 256-byte page, so the high-order byte is a constant. Some of these ideas will likely be possible with intelligent macros, combined with address labels and computation on those labels. I'm puzzling a bit over how to optimize the "same page" condition: whether to include a "link-time assert", which will issue an error if the code is emitted so as to cross a page, or an even more intelligent "link-time computation" which, given the current available memory space, can choose a location that meets the constraints.

That's all still in development. The first test case I used was the classic "Red Book" tone routine.

;; the original
;; location 00: pitch
;; location 01: duration
0002-  AD 30 C0   LDA  $C030 ; whap speaker
0005-  88         DEY
0006-  D0 04      BNE  $000C
0008-  C6 01      DEC  $01   ; y reaches 0: a duration tick
000A-  F0 08      BEQ  $0014 ; duration exhausted when reaches 0
000C-  CA         DEX        ; x counts time to "whap"
000D-  D0 F6      BNE  $0005
000F-  A6 00      LDX  $00   ; x reached 0, reload pitch
0011-  4C 02 00   JMP  $0002 ; and whap
0014-  60         RTS

A few comments:

Locating the code in zero-page needlessly wastes valuable zero-page locations. The code is easily relocatable to other locations.
X counts from the "pitch value" down to 0 at a fixed rate, "whapping" the speaker each time it reaches zero.
Y cycles around continuously at basically the same rate; each time it reaches zero, one tick is subtracted from the duration value. When duration reaches zero, the routine returns.
This classic tone routine has the important feature that duration is absolute, and not dependent on pitch
Y is uninitialized, which presumably causes a slight jitter in the duration, unless BASIC ensures Y=0. Likewise, X in the first pitch countdown is uninitialized.
duration is destroyed by the routine, but pitch is not.

Here is my transcription to COMFY-6502. I have translated the "I", "J", and abbreviated "L", "LI", etc. opcodes for this example, but not yet changed Baker's code to accept them.

(let ((comfy-6502::*symbol-table* (make-hash-table)))
  (equ speaker #xc030)
  (equ pitch 0)
  (comfy-6502-tests::compile-code
    (comfy-6502:loop
      ;; repeat until some clause loses: actually, only exit is
      ;; through return, so each clause should be ensured of winning.
      (LDX pitch)
      (LDA speaker)
      (not (comfy-6502:loop  ;; repeat until whap time
             ;; "whap" time is when a clause loses;
             ;; NOT wins, outer loop continues
             DEY
             (if =0? ; has Y reached zero?
               (not (seq (DEC duration) ; Y has reached zero, duration ticks down
                         =0?       ; if duration has not reached zero,
                         ; SEQ fails, NOT wins, IF wins, inner loop continues
                         RTS)) ; duration reached zero, return
               (seq DEX ~=0?))))))) ; Y not zero, count down pitch.
               ;; If X has reached zero, SEQ loses, IF loses, inner loop exits

To understand the code, the basic COMFY principles must be understood.

Conditional tests (such as "=0?", corresponding to a BEQ or BNE), "win" if they are true, "lose" if they are false.
Ordinary machine operations (such as LDA, DEX, RTS) always "win"
SEQ sequentially each enclosed form, until one "loses", in which case SEQ "loses", or until they all "win", in which case SEQ "wins."
LOOP executes each enclosed form, until one "loses", in which case LOOP "loses", continuing at the top of the loop if every form "wins." (I have introduced an "implied" seq to Baker's ELisp implementation.) It is SEQ stuck on potentially endless repeat.
NOT executes the single enclosed form, but changes "win" into "lose" and vice-versa. This is useful to keep a nested "loss" from terminating all the enclosing forms.
(IF test win-form lose-form) executes TEST. If TEST "wins", WIN-FORM is executed, and its win/lose result is passed out as the result of the IF. If TEST "loses", LOSE-FORM is executed, and its win/lose result is passed out as the result of the IF.

Note that I have chosen to initialize X on the first pass through the loop. That makes the code somewhat clearer, and I think is better style.

So what code does COMFY (as of today) generate for this? Behold.

(; reload-pitch
(LDX :ZERO-PAGE) (:ZERO-PAGE 0)
(LDA :ABSOLUTE) (:ABSOLUTE 49200)
; spin
(DEY)
(BEQ) (:BRANCH 6) ; duration-tick
(DEX)
(BNE) (:BRANCH 8 ) ; goto spin
(BEQ) (:BRANCH 9) ; goto reload-pitch
; duration-tick
(DEC :ZERO-PAGE) (:ZERO-PAGE 1)
(BNE) (:BRANCH 2) ; goto-spin
(RTS)
; goto spin
(JMP :ABSOLUTE) (:LONG-BRANCH -14)
; goto reload-pitch
(JMP :ABSOLUTE) (:LONG-BRANCH -22))

This is a symbolic "intermediate" form, not yet relocated and linked. Some keys for interpretation.

(BEQ), (JMP :ABSOLUTE) etc. represent one-byte opcodes, with the specified addressing mode (redundant in the case of branch instructions, which have only the branch-relative mode).
(:ZERO-PAGE 0) is a single-byte, value 0, tagged to indicate that it represents a reference to zero-page location 0, and is not just a immediate argument of value zero.
(:ABSOLUTE 49200) is a two-byte reference to the address #xC030, which will be emitted in the usual 6502 low-byte first as #x30 and #xC0.
(:BRANCH n) represents a short branch. It is one byte long. Note that the destination counts relative to the BRANCH byte, not relative to the next instruction, as the 6502 branch values do. (The relocater corrects for this when converting to 6502 code bytes.)
(:LONG-BRANCH n) is a long branch. It is two bytes long. JMP is absolute in the 6502; the relocated will deposit the absolute address by adding this value to the location of the first byte of this pair.

I have added this intermediate form because Baker's Elisp goes directly to bytes in a fixed memory area, starting from high addresses. My goal is for a high level "COMFY module" macro to result in an intermediate form that knows its length in bytes, what zero page locations it uses, and what internal entry points it exports. Otherwise, it is relocatable by the linker. I would also like the feature that the zero-page references are symbolic (e.g. (:ZERO-PAGE duration)), in which case the linker would allocate a byte in zero-page, and all modules referring to "duration" would refer to that address. Even nicer would be the ability to do math, for instance (:ZERO-PAGE (+ duration 1)), or to emit (:ABSOLUTE (- tone-entry 1)) to encode the module entry point in the "off-by-one" method used by the "fake RTS" trick. Similarly, one could have elements such as (:BYTE (HIGH tone-entry)), representing the high-order byte of the tone-entry address, and allowing arbitrary link-time math.

As for the code itself, it has ~~the same~~ similar semantics as the Red book routine.[UPDATE: but not quite. See more recent post for correction.] It has a somewhat minor change in that the DEY is followed by a BNE in the original code (taken 255 out of 256 times), and by a BEQ in the COMFY version (taken only 1 out of 256 times). Any difference in the execution time will, of course, change the pitch emitted. More serious is the fact that COMFY, as coded, is biased toward forward conditional branches. The two JMP instructions after the RTS are reached only through branches that could reach their destination directly. COMFY compiles, as I mentioned, from the last instruction toward the first. Forward branches are trivially resolved, but backward branches don't know their destination, in particular, whether the destination is a short branch away or not. The backward JMPs are created by the code for LOOP, which first emits a "placeholder" absolute jump, compiles the loop forms, including branches that might land on that jump, then patches the JMP destination to point to the first form of the LOOP.

That suggests that LOOP might be enhanced to include a "loop-tightening" phase which examines the body of the loop for forward branches (:BRANCH x) to jumps (:LONG-BRANCH y), and if x+y+1 is within range of a short branch, replaces it. (The +1 is for the JMP opcode itself.) The result would be

(; reload pitch
(LDX :ZERO-PAGE) (:ZERO-PAGE 0)
(LDA :ABSOLUTE) (:ABSOLUTE #XC030)
; spin
(DEY)
(BEQ) (:BRANCH 6) ; duration-tick
(DEX)
(BNE) (:BRANCH -5) ; spin
(BEQ) (:BRANCH -12) ; reload-pitch
; duration-tick
(DEC :ZERO-PAGE) (:ZERO-PAGE 1)
(BNE) (:BRANCH -11) ; spin
(RTS)
(JMP :ABSOLUTE) (:LONG-BRANCH -14)
(JMP :ABSOLUTE) (:LONG-BRANCH -22))

The two JMPs are now unused. This is a special case, because RTS is itself a jump. Because LOOP knows that it freshly emitted the JMP, no external code can properly reference it. If all the branches to it have been patched, and the last instruction cannot reach it, the JMP itself can be omitted.Assuming I can convince COMFY to be smart enough to omit the (proven) redundant JMPs, we are ready for comparison. Let's re-write the original code to match the explicit X initialization, and use COMFY's discovery that moving the LDX to the front of the loop means the absolute JMP $0002 in the original can be replaced by a BEQ.

reload-pitch LDX PITCH
LDA $CO30
spin         DEY
BNE not-tick ; actually, takes branch more often
; duration-tick
DEC duration
BEQ return
not-tick     DEX
BNE spin
BEQ reload-pitch ; save a byte compared to original JMP
return       RTS

COMFY, with the yet-to-be-implemented loop tightening has made the code one byte shorter, and relocatable.As mentioned before, COMFY has luckily or unluckily chosen the branch after DEY to be taken infrequently, while the author of the tone routine chose it to be taken frequently. We cannot know whether the author de-optimized to reach a lower pitch range. If so, to match the timing more closely, I can reverse the sense of the IF, causing it to emit a BNE. In this case, the hypothetical loop-tightening can eliminate one JMP, but not the other.

(; reload-pitch
(LDX :ZERO-PAGE) (:ZERO-PAGE 0)
(LDA :ABSOLUTE) (:ABSOLUTE 49200)
; spin
(DEY)
(BNE) (:BRANCH 6) ; not-duration
(DEC :ZERO-PAGE) (:ZERO-PAGE 1)
(BNE) (:BRANCH -6) ; spin
(RTS)
; not-duration
(DEX)
(BEQ) (:BRANCH -15) ; reload-pitch
; goto-spin
(JMP :ABSOLUTE) (:LONG-BRANCH -12)

We see this simple change affected the DEY test, but also affected the DEX test, biasing it to be taken less frequently. Perhaps an annotation could added to the COMFY language to indicate "please prefer (or expect?) this test to win/lose." This would disrupt the tight recursive elegance of the GENBRC routine, but it might prove handy in modern architectures with their branch-prediction tendencies. It is also not so simple to change the test after DEC (:ZERO-PAGE 1) to BEQ to the RTS. That would require moving the RTS to be emitted at the end, i.e. compiled by COMFY first. I'm more interested in implementing the loop-tightening logic than attacking that brain-teaser right now.The moral may be that COMFY (plus yet-to-be-implemented optimization passes) can emit code very much in the ballpark of humans, at least for these toy examples. For cycle-level timing, however, I might follow the lead of Sassy and allow for the human to code branches and jumps explicitly. (Sassy's example "boot sector" code actually avoids pretty much all the COMFY primitives; however, the documentation also seems to suggest that loop tightening and similar optimizations have been implemented there.) And I have yet to successfully translate Woz's code into COMFY, probably because I have to understand it first.(Note to self: AVOID the tempting WordPress "code" button. Use the sourcecode tag, although it supports neither assembler nor Lisp.)

Quis custodiet ipsos...

2007-12-18T18:06:00.000-08:00

I don't know what this says about Lispers' attitudes toward automated testing, or toward the RT test framework, but the commonly-distributed versions of the RT test framework did not pass their own self-tests.

The apparent cause is that the introduction of a hash-table and tail-pointer in rt.lisp to efficiently find tests by name and insert new tests at the end of the test list were not matched by the updating of the rt-test.lisp to consistently construct the miniature mock-up test suite. This diff for rt-test.lisp, introduces code to dynamically rebind those variables. The patch, in addition to one putting the self-tests into a separate package, has been applied to the HEAD of the CVS repository for GNU CLISP. Note that changing the package changes the names of the tests themselves, which requires editing the expected output in the tests.

Note how powerful and convenient the dynamic binding feature of Lisp (optional, as it should be, in Common Lisp) can be: while running the individual tests, the test procedure can effortlessly construct a sandbox in which to test out the features of the test suite, without destroying the suite of tests that is being sequenced through, and automatically restoring the state of the outer test suite as each test completes, even if the inner test throws an unexpected error---all with essentially no special effort on the part of the original programmer (except, as we see, to identify all of the state that needs to be shadowed.)

Another interesting mid-level assembler

2007-12-17T14:03:00.000-08:00

While I'm still digesting COMFY in my "ample spare time," a post on comp.sys.apple2 mentioned some assemblers from "Ron" having a similar flavor: SPL for the 6502 and PIC0 for PIC 10F2xx microcontrollers. They are apparently implemented in Python, and inspired by Forth. They emit assembler.

System 7 Accessing Mac OS X file servers

2007-12-03T03:11:00.000-08:00

Using my MacIvory, until now, I have been using an old Mac Powerbook running Mac OS 9 with AppleTalk file-sharing to allow me to copy files from a USB key to the Mac IIfx host running 7.6.1. Trying to access shared folders on my newer laptop running Mac OS X failed, as did sharing in the opposite direction.

Presumably, the problem is that Mac OS X uses a revision of the AFP file-sharing protocol which is incompatible with the version used by the default Mac OS 7.6.1.

It turns out there is a fix for this issue:

Install Open Transport 1.1.2 (a dependency for the next step) on the 7.6.1 Mac

Install AppleShare Client 3.8.3 on the 7.6.1 Mac

The Apple article on AppleShare Client 3.8.3 has more information on this release of the software.

Apparently AppleShare 3.8.8 requires System 8.

Learning Genera, bit by bit...

2007-11-06T19:07:00.000-08:00

Having hacked on the COMFY code for a while in Emacs Lisp, I got the itch to move it over to my MacIvory, which has been waiting for real work to do. I've crafted some tests in the Emacs Lisp using elk-test (trying to add heirarchical suites-of-suites along the way), and wanted to port them to one of the Common Lisp test frameworks (I'm thinking of checking out Stefil, LIFT, and rt, which I've used briefly). While copying the files, I ran into a few issues.

I couldn't figure out an automatic way to handle line-ending issues in the text files. I ended up locating the file on the Mac OS side in the file browser, Mouse-R clicking to Edit the file, then M-%, C-q [LINE key] [Return] C-q [Return] [Return] ! in Zmacs, then saving to a file on the Ivory file system. (I like the offer to create non-existent directories on a save.) There is probably a Copy File(s) command I could have used to move them en masse, and I should have been able to code up a quick conversion routine or Zmacs keyboard macro, but it worked.

Trying to compile the RT framework, I think I had been misled by my experience in the Genera Development Tutorial. The Common-Lisp mode apparently is Common Lisp the Language, 2d ed. What I was looking for is ANSI-Common-Lisp. The confusion cause all sorts of problems with DEFPACKAGE, and the presence of FUTURE-COMMON-LISP made me even more confused. An e-mail on the SLUG list apparently explains the various Common-Lisp strategies on Genera.

[UPDATE: the old link to the e-mail broke, but gmane.org still has it.]

Baker's COMFY: a few notes

2007-11-03T21:22:00.000-07:00

I've been working a bit with Baker's COMFY-6502 code; a few notes of what I have learned so far.

First, a couple of tiny bugs in the genbrc; the code miscounts the size of the branch instructions, meaning that (- l 2) should be a (+ l 2) and so on. I found it handy to enumerate each clause of the cond. Each clause handles a particular case, such as when the "lose" continuation can be reached by a short branch instruction, while the "win" continuation is far enough away to require an absolute jump.

Second, genbrc, genbr, and compile all provide the address of the resulting "continuation" as the return value. This is perhaps clear when one traces out all the recursion, but it isn't explicitly mentioned. One interesting case is genbrc when the "win" and "lose" branch destinations are the same: one could simply emit an unconditional branch to that destination, and return the address of that branch but actually returning the destination works just as well. (In the emit routine, if the continuation does not happen to be the next instruction, the unconditional branch is, after all, emitted, moving the continuation to the front of the instruction stream, ready for the non-branching instruction to be emitted just in front of it.)

As for "upgrades" to the package, I have been focused up to this point on moving the knowledge of 6502 opcodes and addressing modes from magic decimal numbers to symbolic processing. Instead of simply emitting a decimal opcode, I have been changing the code to emit symbolic opcodes, such as (ADC ABSOLUTE), with routines to reduce these symbolic forms to the appropriate opcode, including checking for invalid opcodes. Baker's original code will happily emit opcodes with addressing modes not supported by the chip. Currently, the compiler emits these symbolic codes into the code vector, awaiting a processing step to convert them to the decimal equivalent. Ideally, one would detect invalid addressing modes in the compilation stage, not the post-processing stage.

Some other changes I am contemplating are allowing for symbolic jump destinations, so that object code can be relocated and external labels could be used to relax the restrictions of the current scheme, which requires manually sequencing compilation and storing of addresses.

MacIvory ADB Adapter Protocol

2007-11-03T21:07:00.000-07:00

Working with ResEdit and DeRez, I believe I have traced out the keyboard protocol used by the MacIvory ADB adapter. My adapter is on the blink, probably because of the small screw I found rattling about inside. I hope DKS can bring it back to life, or the PIC-based project I had begun will make its way back to the front burner.

MacIvory Keyboard Support Software

2007-10-09T15:16:00.000-07:00

The ADB adapter has not just given me some help uncovering the behavior of the Rev. C Symbolics keyboard protocol, but also works when attached to my MacIvory. Along the way (and with the help of David K. Schmidt) I encountered some glitches, mostly stemming from my original accidental trashing of the System file DKS installed.

There are a few software components the ADB adapter box needs to convert the Symbolics keyboard to something the Mac OS 7.6.1 (including the "FEP" window which communicates to the Ivory board on a low level) will accept as a keyboard with the right keymapping, and furthermore to something the Genera application will recognize with all the Symbolics-specific keys. Without these components, you will see one of the following two stages of malfunction.

Stage 1: The mouse connected to the ADB adapter works fine. In the Mac OS (including the FEP window), there is total keyboard confusion, with the keys all scrambled. On the Genera side, things are less scrambled, but some keys are missing, such as "e", "t", "u", and "o"; the "o" in fact causes the Mac menu bar to flash. Modifier keys seem absent. If you activate the keyboard control application, Genera seems to crash easily. Peter Paine reported this at the time System 7.5.3 came out.

If you do a custom install of the "Symbolics Keyboard for System 7" selection in the MacIvory software installer, you get the Symbolics Keyboard extension icon at Mac OS startup, and a Symbolics keyboard layout choice in the Keyboard control panel. The situation improves, reaching

Stage 2: The alphanumeric keys work for both the Mac OS and for Symbolics. However, Genera treats a press of the "Page" key as a press of the "Line" key, treats "Escape" as "Select", and ignores the \{ and |} keys, the Hyper modifier key, and most of the large function keys (Function, Refresh, Square, Circle, Triangle, Suspend, Resume, Abort, Network, Local, Select, Line). Kent Pitman reported this after an upgrade to System 7.5.

The solution is to do a full install of the MacIvory support software. I believe this is necessary to get all of the resources installed in the Mac's System file.

I'll try to gather more details on the resources; it would be nice to understand how to make a box that takes, for instance, a PS/2 keyboard & mouse and connects it to the MacIvory, although the Symbolics keyboards and mice may be more plentiful than than the Ivory boards at this point, so it might be mere archaeology. More practical, if ADB adapters are scarcer than Ivory boards, would be a combination of the Apple keyboard (which works well enough for MacIvory use) and a three-button PS/2 mouse with a "partial ADB adapter" allowing it to provide multi-button mouse function to Genera.

UPDATE: I examined the resources, and was able to decode the Symbolics ADB adapter encoding.

Symbolics Rev. C Keyboard Secrets

2007-09-26T03:24:00.000-07:00

The ADB adapter has fulfilled its main purpose: providing an example of hardware which can speak to my Rev. C Symbolics keyboard.

The apparent trick is that the clock pulses must be narrow, without being too rapid (i.e., a relatively low duty cycle.) A 10 microsecond long low pulse on /CLR, followed 30 microseconds after the beginning of that pulse by the clock going low for roughly 9 microseconds, continuing with a clock period of approximately 50 microseconds does the trick. The ADB adapter pauses somewhat between groups of 11 clock pulses, corresponding to the natural "section breaks" of the keyboard mapping; I don't know if that is strictly necessary. My first crude approximation, using bit-bashing for the PIC with timing loops resulted in Caps Lock and Mode Lock that felt a bit sluggish---needing to be held down relatively long to "lock" or "unlock." The overall cycle time may be important in that regard.

In any event, I reconstructed the keyboard mapping from the LMKBD project, although I chose to count pulses from zero, rather than n=1 in that listing.

Unfortunately, this seems to make the PIC EUSART not fit the application: its clock pulses have a symmetric duty cycle. I had hoped to use the serial port intelligence to offload enough of the bit-handling to allow for rapid response that the ADB protocol seems to require. Perhaps I need to look at some of the other peripherals; can I use some pulse-width modulation output to drive the keyboard and the EUSART in synchronous slave mode together?

Baker's COMFY for the PIC?

2007-09-24T23:25:00.000-07:00

I encountered Henry G. Baker's COMFY compiler a couple years ago. (Baker's site contains a text article, a TeX format article, and an Emacs Lisp implementation. The ACM published the two articles COMFY theory and COMFY-65) COMFY is not a very high-level language, in the conventional sense, but is an attempt to provide a clean but simple set of control structures on top of conventional machine code. Baker calls it 'medium-level.'

The resulting compiler is very small; its main task in life is to automate the generation of branch instructions, which is one of the tedious parts of tightly optimizing assembly programs. Yet it allows as well for arbitrary Lisp-style macros.

I found the concept intriguing, but I found the article and the compiler code itself rather obscure. Part of the obscurity is the unconventional names for the 6502 operations, unconventional notation for the addressing modes, and decimal numbers for opcodes (because Emacs Lisp does not accept other radixes). Another is that the implementation is lean-and-mean, emitting code bytes directly into a destination vector---the only output to look at is a vector filled with decimal numbers. The manipulations on the 6502 opcodes take advantage of the low-level bit patterns without explanation. Finally, it uses the term 'continuation', which, no matter how many times I think I understand it, scares me, probably because it introduces a highly abstract term into an area like assembly programming which is relentlessly concrete.

I've been learning about it by going through the code, restructuring a bit as I go. I'm beginning to be impressed by the subtlety of the code. One thing that is just dawning on me is that the return values of the code-emitting functions is as important as the side-effect. (To those of you chuckling, you see how far I have yet to go.) Lisp can hide that from you when it "looks imperative."

My longer term goal is to see if the same technique can be fruitful even within the more restrictive limits of the PIC. In order to get there, I'm going to have to gain confidence that I understand the formal concept of the 'win' and 'lose' continuations, which means, I think, having to come up with legible examples that compile to 6502 code, and use that to firm up my understanding of the compiler. Finally, I'll try to code the 'genbrc' routine for PIC branches.

For the "legible examples" part, I think I will try to break the lean-and-mean single-pass direct-to-binary compiler into a "compile to a vector or list with symbolic 6502 mnemonics" followed by a very simple 6502 assembly pass. A similar enhancement might accumulate a relocation table to allow more flexible linking. (I'm still not used to the idea that the code is emitted starting in high memory and working down.) I've begun abstracting out the addressing mode bit manipulations, and probably will add error-checking to make sure invalid opcodes are not generated by mistake.

One thing that I think that will have to go in the PIC version is the shallow binding mentioned in the TeX column, but not in the text version; without a stack, I'm not sure where to save anything. I'm also just a bit worried that the PIC has so many idiosyncracies (e.g., register pages controlled by special flags, instead of a uniform zero-page) that even a medium-level language doesn't help much.

(I should mention in passing Frode Vatvedt Fjeld's nifty little Lisp code to generate PIC instructions from the bit chart.)

[UPDATE: I should also mention a COMFY-based assember for x86, implemented in Scheme, called `Sassy', although I have never tried to use it.]

Symbolics keyboard to PIC, step 1

2007-09-04T23:14:00.000-07:00

The first step is to connect the Symbolics keyboard to the PIC. I am using the 16F690 which comes with the PICkit 2 starter kit on the low pin count demo board (Microchip DM164120-1). The keyboard cable mates to a 6-pin modular jack (such as Digikey part number 609-1061-ND), which I tacked onto the side of the board. Unfortunately, this mod jack has pin spacings of 50 mil "horizontally" between successive pins, so I bent up the odd pins and soldered the even pins to the demo board.

Mod jack pin	Keyboard function	PIC function	PIC pin marking
6	GND
5	GND (N/C)
4	Vdd
3	Key Data	RX/DT	RB5
2	CLK	TX/CK	RB7
1	/CLR	RC5	RC5

My first task will be to power up the keyboard, code a scanning loop, and try to detect keystrokes for a single key, such as the Left Control Key, displaying the count of key-down events in binary to the LEDs connected to RC0-RC3. If the keyboard is not debounced by the internal microcontroller, the count will not increment cleanly, but I hope to at least prove that I can talk to the keyboard and make sense of what comes back. I'm not sure how much current the keyboard will need, so I might have to power the demo board with an external supply.

[UPDATE: It turns out the standard numbering of modular jack pins is the reverse of the pin numbering of the P1 header inside the keyboard. I've corrected the table above to reflect the standard modular jack scheme. Looking into the opening of the jack, with the locking tab down, pin 1 is on the left side.]

Learning more about PIC

2007-08-27T02:12:00.000-07:00

A few comments about the PICkit 2 learning experience.

There's a slight bit of polish that could be applied to the example code.

A few redundant instructions, such as
- Extraneous operations setting register pages
- In A-to-D examples, the data sheet seems to say the 5 microsecond settling time is needed only when changing the input source, not for every conversion, as in the examples
- For the 5 microsecond interval, a few of those microseconds are covered by the non-NOP instructions before the conversions

In one of the variable-speed rotating bits examples, the carry flag is not explicitly cleared when it needs to be clear. When I modified the example to change the sign of the pot setting-to-delay conversion, I would sometimes set the carry, putting extra bits into the display. Took me a good fifteen minutes to debug that.

I think a few extra instructions are wasted using the W register and an explicit move to the file register when the file register destination could be used instead

I also got bitten by the issue mentioned briefly in the PICkit 2 release note: if you program the examples from the IDE, the IDE forces the line controlled by the push-button low, meaning the push-button examples fail to work, unless you program the device from the dedicated programming application and the .HEX file produced by the IDE.

~~I am slightly disappointed that~~ there ~~doesn't~~ seem to be any visible progress getting the PICkit 2 with version 2 firmware to work under Mac OS X.

Also, I find a real lack of documentation of the PICkit 2 "Debug Express" support for in-circuit debugging. This may be because Microchip doesn't want to undercut sales of the MPIDE 2 hardware. I think, however, one can use a combination of the AC162061 (containing an ICD version of the 16F690) and AC164110 (to convert the ICSP six-pin interface to the modular plug used by the usual ICD interface) to do in-circuit debugging of 16F690 applications with the PICkit 2, at an additional cost of about $50.

I've started sketching out some ideas for interfacing the Symbolics keyboard on a dedicated page.

[UPDATE: Microchip does, after all, document the PICkit 2 Debug Express hardware requirements (scroll down a bit to see what I missed before.) I ordered the combination even before I found the document, and when the backorder is filled, I will be giving it a try myself.]

[UPDATE: Jeff Post announced that he releasing alpha code supporting v2 firmware for the PICkit 2 on Linux/Mac OS X]

Making lights blink...

2007-08-21T22:40:00.000-07:00

My latest diversion is a Microchip PICkit2. The idea is to make something that can understand the Symbolics keyboard I have, and then connect that something to other things that can speak Apple Desktop Bus (ADB), PS/2 keyboard protocol, or USB, in roughly that order. If I can make something that takes a multi-button PS/2 mouse and a Symbolics keyboard and connects both to a MacIvory, I could be set for beautiful three-button mousing and classic keyboarding on my Lisp Machine.

I've mostly traced out the schematic of my Rev. C Symbolics slim-line keyboard (thank god for simple two-layer boards!). Rev. C has LED's in the Caps Lock and Mode Lock keys, in contrast to this picture of the similar Symbolics Rev. B keyboard. Compared to the Symbolics 3600 keyboard schematic, mine is roughly similar, but unfortunately has a 40-pin microcontroller (an Intel 8749H, a member of the MCS-48 family) soldered in. I am certain I could make something work with the simple protocol the 3600 keyboard apparently implements, but if the microcontroller does anything beyond driving the LEDs on and off based on simply counting the corresponding keystrokes, I might be sunk. I've fired off a few questions to people who can hopefully tell me that all of these are plug- and wire-compatible.

As for the 3600, it is pretty obviously a matter of pulling the reset line low to clear the counters, and clocking through the other 127 counter states to read off the switch matrix closures on the 16 X by 8 Y (I surmise that high on the output would mean the corresponding key is down.) I'm not sure how frequently I have to collect the bits to avoid missing keypresses, and what debouncing I might have to do.

The strange thing about my Rev C is that the ICs driving what I guess is the X direction are a pair of 74145 devices with what the datasheet claims are open-collector outputs, and the IC connected to what I guess is the the Y direction is a 7442 with conventional TTL outputs. That seems to mean that scanning an X column with a pressed key causes an open-collector pulling low to compete with the TTL high output on the Y row, except when the Y scan tests that line with a TTL low. What good that does is a mystery. That, and I haven't finished tracing out the part of the circuit with an LM319 comparator. The 3600 schematic drives the X columns (one low, the rest high) and muxes the Y rows (driving ordinary inputs through any pressed keys) to the readout.

[UPDATE: Seems that the keyboard protocol should be identical, so I'm going to forge ahead with the plan, and assume the circuit knows how to work itself.]

[UPDATE 2: fixed the link to the online 3600 schematic.]

[UPDATE 3: I probably should have given more explicit acknowledgment for the inspiration of the project. If I had not seen asciilifeform's schematic revealing the simplicity of the protocol, I'm sure I would have been far less tempted to start this project.]

MacIvory speaks!

2007-08-08T13:32:00.000-07:00

I've only had a bit of time to play with my MacIvory, but already things are getting interesting.

One is getting used to the system administration style: typical UNIX system configuration resides in text files, and the boot process reads them. Genera is different. The state of the system is contained in the memory image ("world"); you change it through various operations---some interactive, some by loading Lisp files---and preserve it by saving a new world as a file, which can be defined incrementally based on an existing world. The boot process involves loading that saved file back into the living Lisp machine. I haven't yet developed the sense to complete a block of tasks before saving the world, and how much change pushes one from incremental to a complete world.

Another was getting the network to speak, and part of that is the timewarp of working in Mac OS 7, an era in which Mac networking was going through the introduction of Open Transport, and an AppleTalk era which Mac OS X refuses to acknowledge. Luckily, I've got a PowerBook G3 that runs Mac OS 9. My first ethernet card didn't work, either because I screwed up some software setting or because the hardware was bad. (More embarrassing disclosure later.) DKS sent me a replacement, which worked immediately. I've only got ChaosNet configured on the MacIvory side at this point, and my ChaosNet work isn't up-to-date on my Powerbook, so I've only seen raw ethernet packets in tcpdump, but my archaeology is already progressing---documentation to follow soon, as well as getting some Python software (with libpcap or Mac OS X kernel packet filters?) to respond to Chaos-formatted Ethernet packets.

As for the embarrasing disclosure, one of the first things I did was try to set the video card to a mode not supported by my Sony Multisync 15sf monitor. Instead of waiting for the change to timeout and revert, I hit Esc or Command-. or some other key which confirmed the change, leaving me with a blank screen. I tried various reboot, PRAM zapping, or magic key sequences to force the Radius Precision Color 24X to a good mode, but none of that worked, and at some point I screwed up the system. Once I hooked up the Dell 2007fp that understood 1152x870, I saw the floppy-with-blinking-question-mark. Recovery without a working network card involved the ancient hassles of multiple-floppy installs.

I have a Kensington ADB multi-button mouse, but all the drivers I have downloaded are either for Mac OS 8 or cause serious bugginess: system freezes or error -192 when launching the configuration application. I hope I don't have to roll my own software to get multi-button goodness to work with the Ivory. (At this date, few vendors pay much attention to 68k Macintosh support.)

UPDATE: I've captured a bit of the tcpdump results in my ChaosNET information page.

Ordering a Lisp machine

2007-07-11T17:03:00.000-07:00

Just sent off the Paypal payment for a MacIvory II system with 2.6 MW of memory. I'm hoping also to get a Symbolics keyboard, although Symbolics is apparently out of the ADB box that allows the keyboard to be used with the vintage Macs.

It was a tough call whether to splurge for this or, for about twice the money, a new MacBook Pro. I'm hoping the Symbolics machine will be much more entertaining.

My main idea of a project is to use the Lisp Machine as a Chaos FILE host, in order to allow development on the CADR simulator with a versioned file system, without having to simulate and learn ITS or TOPS-20. To get that to work, I probably will need to do a little bit of investigation to understand how the Symbolics box gets configured to talk on the network, what Chaos packets look like on the Ethernet, how to get those to my Python/Lisp code on the modern machine, and, for data security what would be needed to talk to a CD-ROM drive on a remote computer, or to use a remote computer to do backups (to a simulated tape drive?). In order to get 2.6 MW of NuBus memory, I had to choose a Mac config without a CD-ROM drive, and I'm not too excited about trying to get an external SCSI drive for this old machine. Backing up to floppies is another thing I don't want to relive.

Some other things I'm looking forward to are Zmail, Concordia (to generate HTML output, anyone?) and Document Examiner, using CLIM, and grokking the Symbolics compiler(s).

My non-blog web site

2007-03-13T11:37:00.000-07:00

I should mention my non-blog site, which exists pretty much to hold information for recruiters who want to get some idea of my software experience. Don't expect much exciting there. I am, by the way, currently and happily employed, but am interested in opportunities in the New York City metropolitan area.

ChaosNET and Python

2007-02-05T12:30:00.000-08:00

My blogging has been much less frequent of late, mostly because of an unfortunate abundance of non-computer issues. Despite those, I believe I have made some progress in my Chaosnet-on-OpenMCL work, but I've switched most of my hobby-development to Python, mostly to get in touch with trends outside the Lisp Universe.

My current main project circulates around ChaosNET emulation; the main thing going for Python here is a socket interface that is roughly compatible between UNIX-like and Windows operating systems. Lisp can support that as well, but not for my favorite and familiar implementations. (One thing I hope to try before I get through this is to get an asyncore-like framework set up in CLISP.)

So far, I have implemented a `chaosd' process in Python that simulates multiple ChaosNET `ethers' in the form of localhost TCP streams (on Windows or UNIX-like) or UNIX-domain sockets (on UNIX-like); each stream corresponds to a subnet. The idea has manifold goals:

Allow the usim CADR simulator to communicate on Windows with minimal alterations using Winsock in place of BSD sockets (Done) (Brad Parker's server, however, works with forking and other UNIXisms that would require more work)

Allow separation between local activity and activity exposed or connected to the wider Internet

Provide 'masquerade' and 'network translation' bridges, to flexibly map external resources, such as an ITS or TOPS-20 simulator into something that appears to be a local network. (The translation here is necessary because ChaosNET is a LAN architecture; outside resources might have their own incompatible scheme of subnetting and addressing. These translating gateways would allow me to place external UDP-responding resources into arbitrary Chaos addresses in my local scheme, without requiring any change on the external resource.)

Eventually, provide small modular `gateway' simulators that appear to be ChaosNET resources on a local subnet, but translate to external services such as AOL Instant Messenger or IRC.

Recreate `historically accurate' network organizations for amusement

If I ever break down and get a real Symbolics machine, I expect it to be able to speak Chaos over IP or raw Ethernet; I would like some way to connect that to multiple simulators on a single Mac or PC

As a step in this direction, I have implemented a stream-to-UDP masquerade, that automatically takes traffic on a local `ether' for a particular address and feeds it over UDP using Bjorn Victor's protocol. Running this on two machines, I am able (mostly) to allow a Lisp machine simulator on my Linux box and another on my Windows box to both believe they are on a single Chaosnet LAN, and to communicate with one another. The next step is to get something like a local ITS or TOPS-20 simulator running in KLH-10 on Linux supporting Bjorn's UDP protocol (or possibly Bjorn's machine) to provide FILE service and e-mail service to my Lisp machine simulators on either machine. I have been learning to recompile ITS, with mixed success. My initial impressions on Python in another post.

Dell 2007FP and xorg.conf

2006-08-07T14:56:00.000-07:00

In order to better serve those folks Googling "2007FP xorg.conf", I've included an annotated xorg.conf file that seems to work well for me and my Dell 2007FP monitor. I hope it helps you. (It is, naturally, provided "as is", without warranty of any kind, etc., etc., because it could very well damage your hardware, although nowadays monitors are much more forgiving than they once were.)

(It seems from my site logs that my venting about my adventures with Linux, xorg.conf, and the 2007FP is one of the more popular Google hits for my site now, just one day after it was posted.)

Venting about Linux

2006-08-02T16:28:00.000-07:00

Time for a "Linux is not ready for the desktop" rant.

Some time ago, I picked up a Slackware 10.1 4-disc set. I've been trying to use it on a Dell Precision 610 MT workstation, Xeon Pentium III. (That's what you can read on the label.) Through some struggling through the initial start up screens, I was able to determine the box has two SCSI internal drives. I also have a Dell 2007FP monitor (pretty nice).

Getting X to come up in something other than 1024x768 was an ordeal. Perhaps I should have realized that no amount of so-called "handholding" was going to help me, and gone to the xorg.conf man page directly. Silly me, I thought this would be like a normal desktop environment, where I would open something called something like a "control panel" and find a sub-thingy labelled "Display" (KDE buries this under Peripheral) and select the 1600x1200 resolution my display claims to support. No. Only 1024x768 or worse.

So I go to slackware.com; the FAQ is skeletally brief. It talks about xorgsetup (must run as root, not just using "su" but "su -"), which dumps a uselessly generic xorg.conf file in /etc/X11. xorgconfig asks me to answer vague questions about my mouse and keyboard EVERY time, and scroll through long lists of video cards, none of which is a close match to what the Dell startup screeen says: "Diamond Viper V770D 32MB" It asks me for refresh rates, but doesn't allow me to specify that "at 1600x1200, this monitor only allows 60 Hz vertical refresh". The slackware page config/x.php says something vague about "run X -probeonly" but "startx" seems only to feed its arguments to xterm, then puke when xterm does not accept it. I cycle through various efforts, copying the result to /etc/X11/xorg.conf, and get a blank screen with the monitor complaining "frequency out of range: use 1600x1200@60Hz."

A search through the the Video HOWTO for Diamond Viper 770 shows me RIVATNT2 XF86_SVGA and nv.

Finally, I break down and edit xorg.conf by hand. It takes a while but I start to realize that various identifiers (described just as "free form" by xorgconf, actually are used to refer to various entities in later descriptions. I hand edit my xorg.conf to indicate my "Monitor" can HorizSync at 30-82, only VertRefresh at 60 Hz, my "Device" has driver "nv", and my "Screen" is that Mointor, Device, with Modes "1600x1200" "1280x1024", etc.

Finally. It seems to work.

Next up, try to get USB flash memory key to be recognized, without following directions which tell me to use the SCSI /dev/sda files that are, ahem, being used for my SCSI hard disks.

[UPDATE: Fixed the link to the Linux Hardware-HOWTO video section.]

Clive Page’s Free Fortran 77 guide

2006-08-01T15:59:00.000-07:00

Clive G. Page's Professional Programmer's Guide to Fortran 77 is available under the Gnu Free Documentation License. I suppose this means I should dig up my texinfo translation of his LaTeX, which I did a number of years ago, merge in his more recent changes, and release it.

More adventures in multithreading

2006-07-31T10:39:00.000-07:00

Just wanted to capture a few more thoughts; over the weekend, I experienced what I think was my first case of deadlock. My chaosnet-on-OpenMCL implementation had worked just fine when the two sides of a connection were careful to exchange only one packet at a time, taking turns. When I tried to listen to a SEND (the Lisp machine analogue to an instant message), the CADR emulator sent two packets in quick succession, and my OpenMCL process would freeze trying to receive it.

As an aside, I get the feeling that I don't really know how to debug this kind of thing in OpenMCL: looking at backtraces does not always make it completely obvious which semaphore or lock is being waited for in the various threads, and I don't think it is possible to look at semaphores to see which thread(s) are waiting.

In any case, I concluded that my somewhat haphazard use of a "count lock" to protect access to packet number information in a connection was either being used recursively, or violating a lock heirarchy. So, after some time trying to apply my powers of deduction, I decided to look at the ChaosNET problem from a high level to determine what locking is actually necessary.

The main complexity has to do with the transmission window. In theory, it seems like an ideal case for a counting semaphore: the send window is a limited resource, and sending processes must wait until a spot within the window is available. (Generally, one would wish to preserve FIFO order, but strictly speaking, multiple threads trying to transmit on the same channel might not deserve any guarantees that their packets will happen in any particular order.)

However, there are several things unusual about the window: the thread handling incoming packets can get both acknowledgements in ordinary packets, as the receiving user process consumes packets from the window, and in STS packets that could, at any time resize the window. In the usual semaphore model, it is not easy to deterministically reduce the maximum level of the semaphore. I coded an unsatisfactory approach which does a trial-wait-on-semaphore for the number of window packets that have been taken away. But these might not succeed, if the window is filled above the new capacity. In that case, the window must presumably be allowed to drain below the new level before any waiting process gets allowed to put packets in. It might be possible to have a separate integer variable tracking the "excess window consumption," and for processes to loop back and wait again for the semaphore if the window has been "tightened" since they first waited. I don't know if it is possible for threads to maintain their original FIFO order in this case, or whether it matters. (I'm not sure it is reasonable for Chaos processes to reduce the window, once it is declared in the initial STS negotiating the connection, as opposed to widening it to allow for better throughput. The AIM memo is silent on this.)

Interestingly, the draft chaosnet memo does discuss enlarging and shrinking the window, and mentions WIN and WTS packets.

In terms of avoiding deadlock, there are many participants: the user reading process looks at the window to see if the connection might be getting "stymied", and needs an STS to be transmitted. When STS or acknowledgements come in, the retransmission of the receipted or acknowledged packets should be suppressed, and the acknowledgements (but not receipts) open up space in the window. In the Lisp Machine design, the receive-sts et al. use the RECEIPT function to trim the send-pkts list. I felt that contending with the sending/retransmitting process was asking for trouble. Therefore, I chose to have the retransmission process look at the receipt and acknowledgement status to clean up the send-pkts while it needs exclusive access to the send-pkts list anyway. The retransmission process needs access to the send-pkts list to retransmit, and to the counters if it is required to do clean up. Furthermore, updates of the acknowledgement and receipt states should rachet only forward, i.e., a stale packet coming through the pipe should not erase the acknowledgement or receipting indicated by the logically latest packet received up to this point. That requires locking to protect the read-modify-write update of those status numbers.

One interesting gap in the ChaosNET documentation is whether retransmitted packets can or should update their acknowledgement and receipt information to reflect what has happened to the connection in the meantime. The Lisp Machine implementation seems to retransmit the packet byte-for-byte. An alternative would be to update those packets. In practice, I suspect retransmission should be conservative; it is necessary when the receiving process has fallen behind, or the channel is experiencing loss, and the bias should be toward more retransmission than strictly necessary, instead of hoping to "make up for lost time" by advancing the state of the return channel.

I have created enough of a system to work with the SEND transaction, except for one detail: in a Chaos "stream", my functions indicate an EOF condition only when the request for a byte has gone "off the end." I felt the EOF detection should predict whether the next byte request should succeed; but I could not remember how this works elsewhere.

Apple II Disk Transfer

2006-07-25T10:11:00.000-07:00

It seems that others have also had the desire to upgrade the ADT (Apple Disk Transfer) tool to work with disks other than the Disk II 5-1/4 inch floppy format under DOS 3.3.

ADTPro is a ProDOS-based version of ADT. It should presumably work with all block devices recognized by ProDOS. I e-mailed some code to David Schmidt which allowed ADT to work with the ordinary IIgs serial ports; direct access to the Z8530 chip might allow for faster transfers; that's a potential project.

More thinking about multithreaded queueing

2006-07-19T09:44:00.000-07:00

Looking back on my previous post about multithreaded queues, and based on a little hacking I was able to do last night on my multithreaded-Lisp-user-mode-Chaos-client, and having read a bit more in the Little Book of Semaphores, (must stop typing 'sempahores'!), in particular the turnstile pattern, and the barbershop problem, there are a few other ways to slice the onion.

A "waiting room", i.e., the privilege of blocking on the queue is exclusive. That actually makes a certain amount of sense when, for instance, Chaos packets are already sorted by the destination connection. Does it really make sense for multiple consumers to be concurrently transacting on a single connection? What would make sense, in a C10k kind of way, is rapidly generating server connections in response to RFCs, and handing these connections to waiting server threads. Or, if a connection is known to have independent incoming packets, or is deliberately recycled, connections can queue through a turnstile, each take one incoming packet, then go off to chew on it. I think the mechanism could be a simple lock.

"Signal enough" can be expanded to include a "turnstile" approach, where a disappointed consumer signals the semaphore on his way "out" so that the next waiting consumer can be disappointed as well. The idea that the reason for the disappointment should be visible to incoming consumers before they block is really optional. The main reason to prefer it is that, if we believe the "disappointment" is permanent, that we will eventually want to discard the queue, but we presumably cannot do so if consumers can continue to block. That is, if the disposal process must race against the arrival of new consumers. Discarding a queue when the semaphore still indicates the presence of (non-existent) packets and no threads are waiting is OK. Discarding a queue when the semaphore indicates no packets, but threads are blocking requires a well-defined mechanism for blocking threads to cleanly deal with a "killed" semaphore. I.e. the "abort" mechanism.

I really like the Little Book of Semaphores. However, I find it hard to be sure that I truly grasp the concepts. I can read through the solutions to the problems, and understand the discussion, but that doesn't give me a huge amount of confidence in my own solutions to slight variations on the problem. Maybe this kind of programming is simply hard.

I've tried casting some of the solutions in OpenMCL Common Lisp. It's hard to verify that things don't work by accident. One thing that wasn't particularly clear to me is whether it is kosher to do things like

(let ((semaphore (ccl:make-semaphore)))
   (labels ((client-function () #|...|#)
             (server-function () #|...|#))
      (let ((thread1 (ccl:process-run-function #'client-function))
             (thread2 (ccl:process-run-function #'server-function)))
         (#|...some way to wait for both threads to terminate..|#))))

It seems to work, except I haven't yet tried to make a clean "wait for both threads to terminate", and have just waited for the length of a global list to get to a certain point, or similar kludges. The idea, however, that the let can create "global" variables, but the lambdas can have "thread-local" storage, seems not to be formally specified.

Some old thoughts about TeX

2006-06-27T22:32:00.000-07:00

I saw a link on reddit the other day regarding some math-related videos, including a series of lectures by Don Knuth, including some on TeX internals. (One frustrating bit about the Web video is the blurriness of his terminal. Another is that he uses what is now a slightly-out-of-date version of TeX itself).

He made a comment near the beginning of section 3, probably similar to things he has written, to the effect that in his education he had benefited both from reading badly-written programs, because they were evidence he could do better, and from well-written programs, because they were a pleasure to read. He then wondered aloud which category TeX is in, but that either way it was a win. I'm not at all sure myself. Perhaps TeX is a monument to the best that can be done in Pascal, and simultaneously a warning that it is the best that can be done in Pascal.

I had a definite feeling during grad school, when I was learning LaTeX to write my thesis, and learning Lisp on the side, that a large chunk of TeX internals was spent managing things that manage themselves in Lisp

Dynamic memory (including variable-length strings and reference-counted objects)
Interned static strings (i.e. symbols)
Rational arithmetic protected from overflow and portable between machines
Heterogeneous lists

Knuth also spent a good deal of time, however, fretting about the "inner loop" of TeX, so perhaps re-implementing it in a higher-level language would incur a performance penalty of large constant factors in the runtime of TeX.

I spent a few odd hours with TeX, The Program, coding a few bits in Lisp, as well as beginning a few sketches that could serve as commentary. (One of the most grievous flaws in TeX, The Program is its utter lack of diagrams, apart from the rough memory map on the back flyleaf.) The main topic of the commentary was to sketch memory structures and the overall memory map in diagram form.

Another grievous flaw I saw, perhaps related to the efficiency concerns, was the monolithic architecture. He speaks of TeX's eyes and mouth and digestive tract, but the WEB presentation blurs this all together; as far as I could tell with my limited reading the TeX syntax is processed directly into lower level memory structures without any clear documentation of how those memory structures should be thought of as logical constructs. That is, there is no higher order description of how TeX should behave in the code, simply procedures to take TeX and produce almost raw bytes, and then slurp up those raw bytes into DVI output, with certain very carefully designed algorithms for math layout and paragraph breaking hidden inside.

Knuth's approach to the lack of abstraction in the Pascal expression is to add English prose, both in the WEB format of the code, and in the TeXbook's user documentation, but not to present a more abstract, formal, or diagrammatic description of the architecture. (He does describe some small algorithms in more precise ways, such as the DVI optimizations using down_ptr and right_ptr.) I would hope to replace this by an elegant Lisp description of the underlying structures and algorithms, leaving the messy hand-made arithmetic and memory management off to the side as part of the unfortunate need to run on 1982-era platforms. But I get the nagging feeling that the whole design of TeX the language and the data structures and the algorithms is just one big ball of mud that will resist such analysis.

I am slightly annoyed by the use of numerical coincidence (e.g. using the numerical ordering and grouping of an enumeration to simplify some decision logic) and a few related hidden limits (must get my copy of TeX, The Program to see where I wrote one in the margin). Seems rather too low-level for someone trying to present things clearly.

Another one of my past TeX-related projects was to try to get a from-scratch Pascal-based WEB system producing TeX on Mac OS X, so that I could really understand the WEB code, e.g. by enhancing the TeX internal debugger, instead of simply using a Web2c translation.

link dump for 22 Feb 2006

2006-02-23T11:42:00.000-08:00

A bunch of stuff I was reading back then

Weak Data Structures write-up by Bruno Haible, courtesy of Gary King.

The classic UNIX HATERS Handbook.

Panda's TOPS-20 System.

Spare Time Gizmos store front to buy actual hardware emulators of the PDP-8.
Rainer Joswig e-mail on feel of old Symbolics Lisp Machines

An e-mail discussion about CHAOS vs. TCP in port collisions, a list of Symbolics CHAOS services

Google cache of mahalito
Symbolics Y2K statement
ITS site HACK directory

Google Groups discussion on Basic Chaosnet information
Google Groups discussion on Chaosnet and ARP
Google Groups discussion on Transmitting Chaosnet over Ethernet
Google Groups discussion on Why Chaosnet (nostalgia)
Google groups search for chaosnet

ITS build information build.doc.txt

36bit.org
Retro PDP-10

Some ITS links

2006-02-16T12:03:00.000-08:00

Wanted to collect a few links on ITS, the Incompatible Time-Sharing System

The ITS 1.5 Reference Manual

Bjorn Victor's Luser's Guide

Bjorn Victor's e-mail setup (perhaps useful information for Lispm Zmail setup)

The KLH10 PDP-10 emulator

More information on PDP-10 emulation

The www.its.os.org archive of AI & MC

Paul Svensson's ITS system on the Web

Some ChaosNET thoughts

2006-02-14T07:21:00.000-08:00

I looked a little more closely at Bjorn Victor's (TODO: add diacritic to o in Bjorn) work on Chaos emulation for ITS emulators, particularly the CHUDP protocol for passing Chaos packets through UDP.

Apparently, it uses a simple header (different from the user-mode UNIX header), and dynamically adds to its routing table when it receives incoming packets. This seems particularly handy.

Also, I noticed that the chaosd/server code for the user-mode UNIX implementation does not typically check the hardware destination, but jumps right to the software header for the destination information. This seems wrong.

I'm trying to figure out the most useful way to improve the Chaos support. Some of the use cases would be

use Internet hosts as repositories for CADR Lisp Machine microcode and "world" bands

use Internet hosts as file servers for Lisp machine source code

provide Internet-based early-21st-century substitutes for 1980's functionality, such as e-mail and instant messaging (e.g. get Zmail to use Google's Gmail service, use Jabber or AIM to support instant SEND messages, use NTP to provide TIME service)

Some of these would simply be a local process posing as a Chaos server which could respond to various contact names with useful functionality, configured by various users to, e.g., use the instant messaging protocol or e-mail provider of choice. Some of this would require or allow collaboration between nostalgists: someone volunteers, similar to the ITS community, to host the equivalent of AI, and provide ITS-like FILE service on an IP address & port, with a corresponding Chaos subnet/host address.

I suppose part of my problem is I don't understand all the issues involved in internetworking. Chaos addresses were supposed to be roughly "institution-wide" with specific gateways to other networks, advertised by response to a particular Chaos contact name (such as ARPA). How does this translate to isolated hosts on the Internet posing as institutions with Chaos servers? Can we replace it with a DNS-mediated free-for-all where various IP-based users advertise Chaos DNS entries? How would subnets be allocated? Or will we depend on strictly local, handmade routing tables, where each chaos address gets mapped to an IP host & port? In this case, how do the Chaos source and destination addresses get fixed up when they make it to the IP host & port? Can the sender and receiver basically hide their respective internal Chaos subnetting schemes from each other?

What’s in a name?

2006-02-08T23:57:00.000-08:00

I started this blog with a perfectly cryptic name: "wpblog." I suppose it is a natural impulse shared by many people who download WordPress and are faced with a blank box. I spent an additional minute on the issue today, and decided this was a blog about "getting down to brass tacks." Hence, my current name "BrassTacks." I'm not sure how good Google is at measuring the "uniqueness" of this sort of thing. "Voltaire" as the web host was a nod to my previous blog impulse left unfulfilled at blogger.com, when I was hoping to develop a vicious talent for satire. The name alone proved ineffective, but easy to remember.

It turns out I have a few good names stored up; one is a nom de guerre, which, for security reasons, I will keep secret. I also have here a good name for an instrumental group: "The Cooper Brass Choir." Properly euphonious. If you have a good enough name for a techie blog, you may have it in exchange.

What is this about

2006-01-29T18:39:00.000-08:00

This blog is a simple experiment for now, to understand what is available for a "scratchpad on the Web."

I have a picture which is something more like a lab notebook for my various hacking projects. The requirements are roughly

access from various places where hacking might happen; an idea for a project can happen at work, but I don't want my personal laptop with me at all times.
some kind of permanence, as suits a lab notebook. Various things this might mean:
- version control with timestamping
- no revisions allowed at all (this seems extreme: one wants to separate a polished presentation view from the underlying archival lab notebook. One can revise slides for a talk without altering the historical record of the lab notebook)
- ability to archive into a big .tgz for burning to disk as necessary
flexible project documentation
- easy way to archive associated files; diffs, error messages, screen shots, sketches, links, data sheets downloaded for the web.
- perhaps I need to wait for my copy of Kanare's Writing the Laboratory Notebook (as seen on Lispmeister) to arrive to understand what should be required here.

Some projects to comment on here

work on CADR emulator and MIT-released Lisp machine code
- make it work on Mac OS X/PPC for my laptop
- make Chaos emulation work on Mac OS X
- fix some Y2K-type issues, site configuration
- create Chaos layer in OpenMCL or more portable Common Lisp, to develop gateways to things like
Apple II file transfer (extend ADT = Apple Disk Transfer) to include 140k, 800k ProDOS and Macintosh HFS floppy formats
Possibly, an Apple II compatible USB peripheral interface, to allow easy transfers to a Mac/PC with USB connection
Mac OS X support for foot-pedal shift & mouse-click operations

Interesting that [for Wordpress] Netscape 7 on Windows gives me little editing widgets on my post that Safari on Mac OS 10.4 didn't give me (as I recall.) Perhaps I should check, try to add that capability.