If you've used any of the software or information I've provided and feel so inclined, you can leave a tip for me on PayPal. And of course, I'd like to hear about successes or problems you've had with any of it. Email me!
readme.hyc Change history, another text file
MiNTnet OVL 1.8604, src & binary Fully updated, with OpenSSL 0.9.6 support (1.8603 has been deleted, it was too badly broken)
cabssl.ovl.gz Relinked with OpenSSL 0.9.6c. I have rewritten the OpenSSL crypto math library in 100% assembly language. This binary is 5 times faster than the previous one for SSL connections. 2002-02-18
1998-02-13 Note: If you have an accelerator such as Nemesis or AfterBurner and have experienced frequent crashes, try turning off the Fast RAM flags on the CAB.OVL binary. This should make things much more stable. I don't have any of this hardware, so I can't investigate the problem myself.
back-ldap: An LDAP backend for slapd. I developed this with the pre-1.2.2 release of OpenLDAP. It has long since been incorporated into the OpenLDAP source tree, so I have removed this archive.
My friend Andy Gough just reminded me of the game yesterday (2006-08-15) and
I dug up my C/curses version of the game. I think I finished this version in
1990, and it actually has the full game implemented. The
C source compiles on Atari/MiNT and on Linux; it should work on most
Unix systems. Instead of function keys it uses the numeric keys and the row
of letters underneath for command input. Use the '-' key to display the
list of commands.
Parallel Make
Yet another bit of code I've spun out into the world - I invented the parallel
build scheme in GNU Make 3.78. A brief overview of the story is over at
Paul Smith's web page
. I actually had prototyped this design back in 1991, and submitted it to
the GNU Project back then, but it apparently was lost. (The original work
was part of my investigation into using multiprocessor supercomputers as
parallel compile servers, which I presented at the Sun User Group Conference
in December 1991). In the meantime, the maintainership of GNU Make changed
hands, I had changed jobs twice, and it didn't resurface till 8 years later.
PS: If you still have a copy of the December 1991 SUG Conference Proceedings available, I wouldn't mind getting hold of it. My copy seems to have disappeared and the SUG itself has long since vanished.
PPS: A friend sent me a copy of the paper, and I've reformatted it into HTML here. 2008-11-05
I originally released this as fncchk14hyc but in the interim I've made enough significant changes to warrant assigning it a new revision number. I released version 1.5 on 2001-12-10 which included mutex locking to help with thread compatibility, and heavily optimized memory trace processing. New in version 1.5.1 is more accurate thread support and a gprof-style call graph output. Also the memory trace processing has been tweaked to be even less memory-hungry than before. -- hyc, 2001-12-14
Not much feature-wise has changed in 1.5.2 but it now works on Solaris. It no longer depends on glibc's malloc_hook mechanism for memory tracing; instead it intercepts malloc directly and uses dlsym to forward the request onto the real malloc functions. This feature should be portable to any system that supports SVR4-style libdl. The pthread support is still a hack though. -- hyc, 2003-03-14
Fixed some problems in fncdump on Solaris for 1.5.3. Also tweaked the time accounting for threading, should be more accurate now. -- hyc, 2003-12-03
I ran into some portability issues building this on my AMD64 box, those are now fixed in the 1.5.4 release. Also I finally added something I was missing from using gprof - it records the time spent on each individual call arc, not just total time in each function. -- hyc 2005-07-04
ElectricFence
The ElectricFence library written by
Bruce Perens
has been a very useful malloc debugging tool as well, but it seems Bruce
stopped supporting it back in 1999 and it doesn't work with threaded programs.
(It will deadlock almost immediately.)
I wrote a patch for ElectricFence 2.2.2 that fixed the reentrancy problems
and added pthreads support and emailed it to Bruce but never got a reply.
So here it is:
Patch for ElectricFence 2.2.2 -- 2001-11-25
Malloc Benchmarking
So not only is malloc a pain to work with when you have to chase down
memory leaks, it's also pretty expensive even when you're using it
correctly. We never really knew just how expensive it was, we just knew
that we should avoid it unless it was actually necessary.
I've moved the writeup to it's own page. Click here
for the rest of the story.
Asus ChkMail Email LED Program
I just bought a new Asus M6Ne notebook computer. It has a lot of slick
features, including an LED that is supposed to light up when you have
unread email in your Inbox. Unfortunately the program that Asus provides
to control this LED only works with Microsoft Outlook or Outlook Express,
and I've recently migrated from Outlook to Mozilla. So I've taken it
upon myself to fix this problem. I've written a small program that will
work on Windows XP, with any XP-enabled email client. Remove the Asus
ChkMail from your Startup menu and run this one instead.
chkmail.zip -- hyc, 2004-07-28
By the way, for a program that simply toggles the Email and Wireless Network LEDs, try this leds.zip
Centrino Memory Timing Tweaks
Even though the i855 chipset used in Pentium-M laptops only supports
up to PC2700 DDR memory, not all memory is created equal and you
can get faster performance with higher speed memory. Info here:
Centrino Memory Speed -- hyc, 2004-09-15
Caller ID Reader
I hate getting interrupted by the phone when I'm working on the computer.
(That is, just about all the time.)
There's plenty of commercial and shareware programs out there to display
caller ID info on your screen, but I also hate paying money for software
that is braindead easy to write. So anyway, here are two tiny programs I wrote
to solve my problem: cidsrv.c and
cidout.c. You need to have a modem that supports
caller ID attached to a computer (and phone line with caller ID service)
to use this code. The cidsrv program
is written for Linux, since my Linux server is on 24/7. It opens a tty
device with the modem attached, resets the modem, and tells it to pass
raw caller ID info to the computer. Then it sits in a loop, reading from
the modem, looking for caller ID data. Whenever it receives any, it
packages it up in a UDP packet and broadcasts it to the network. The
cidout program will listen for the packets and print them as text.
So now I leave the cidsrv program running on my Linux server, with the cidout program running on my laptops. As long as I'm in range of my wireless router, I can see who's calling without having to look away from my laptop screen. No more annoying distractions while I work. Also since the info is broadcast to my whole network, I can view it from any of the other computers around the house (including the one feeding video to my TV - no more major interruptions during movies now either).
The code also serves as a very basic demo of how to use UDP datagram sockets. It also demonstrates very clearly the principle of separating data generation from UI formatting. Some folks refer to this as "client-server computing" but that's really useless for describing real computing models. In this case, both cidsrv and cidout are servers. By the way, I got the Caller ID data format info from this Home Automation web page. -- hyc, 2005-09-13
I added support for the D-Bus Notify service on Linux. Most docs discourage you from using the plain C D-Bus API, as it is rather tedious. But all of the higher level APIs are such pigs I decided to try it anyway. The program's VSZ is now 3 times larger than it was, just using plain D-Bus. The code is in cidnot.c.
I used version 0.9 of the
Desktop Notification Specification along with the
D-Bus Documentation
to get this working.
-- hyc, 2010-04-18
Unfortunately, my changes are only in the 1.8 Mozilla source, which will
never be publically released, and some changes are in Thunderbird, but
you have to install a variety of extensions before they get enabled. Also,
as of today the 1.8b2 source is broken, two lines of code needed to update
the screen after a header download were deleted and haven't been re-added
into the source yet.
It's nice not to be stuck with Microsoft, and it's nice to have full
source code for a tool that I use so that I can make it work how I wish,
but it's still frustrating to see that the project itself is basically
dead, and it's also frustrating that I have to wait so long to see accidental
code omissions corrected. Certainly we don't drag things out this slowly
on the OpenLDAP project. Oh well,
that's life in the big Open Source world.
A lot of fanfare has been made over FireFox, which is as far as I can tell
just a lobotomized Mozilla web browser. With a project whose roots are a
tightly integrated application suite providing "every" internet tool you
might need, this strikes me as a huge step backwards. Maybe, just maybe,
not everybody who needs a web browser needs an email client and news reader.
But certainly, everyone who uses email and news actively needs a web browser.
HTML email is probably not the greatest invention of recent history, but
it's a fact of life now, and an email client that doesn't have instant
access to a decent HTML rendering engine is basically unusable.
I'm not sure what prompted this rant. Just needed to vent a little. Firefox
and Thunderbird are crippleware, IMO. The Mozilla Suite is still the tool
of choice; if only they would release the 1.8 version so that it is
actually useful. Head on over to
Mozilla and check out the 1.8 beta. If you're a POP3 user, you'll
be glad to have it. -- hyc, 2005-04-20
Another thing that bugged me while I was hacking on Mozilla was how
incredibly slowly this code built on my Windows system. It turns out a
lot of the overhead was because the Windows build system was built
around Cygwin, which is notoriously slow. As I got into beating the
MailNews POP3 client into shape, I also converted the scripts and Makefiles
to use MSYS instead of Cygwin. I contributed this work back to the
Mozilla project as well, and it slowly got integrated into the
main tree. (See
Bug 294122 for some of that history. In one case over a 3:1
improvement in build time was reported.)
Despite the fact that MSYS works so much better, the Cygwin build was
still the official build platform. But a mere 19 months later,
plans
were made to drop Cygwin. And finally, almost a full 2 years later,
the switch has been flipped and
MSYS is now the
recommended build environment.
It amazes me how much pain other
developers are willing to tolerate, putting up with build times 3
times slower than they need to be, and the commensurate loss of
productivity while waiting for new code to become ready to test. That's
a 3x loss of productivity compounded over a span of several years. Just
unbelievable. And the flip side of this is, now they'll see a boost in
productivity; a boost they never even knew was possible until I came along.
Some of the folks who recognized my name asked "what's your
interest in this code, I thought you work on OpenLDAP?" By now it
should be obvious - I work on whatever I need, to get my work done.
Working on OpenLDAP, I've tracked down and fixed bugs in Cyrus SASL,
OpenSSL, BerkeleyDB, Heimdal Kerberos, autoconf, libtool, glibc, gcc,
and every other tool or package we rely on.
I've always had a policy to only use software that I contributed to.
I think it's important in the world of free software - when you get value
out of something, you have to give something back. It's important in
life in general, but the expression of that ideal is a lot more
clear in software. Over the past twenty years, I've used a lot of
cool software. And improved all of it in passing. The world may not
always recognize the difference, but at least I know I've done my part.
-- 2007-03-02
Mozilla Hacking
I got roped into using Microsoft Outlook at a former job, and continued
using it for a while, but got really fed up with its ridiculously slow,
single-threaded POP3 client. I went looking for something better, but
nothing worked quite how I wanted, so I downloaded the Mozilla source
and started fixing it to my tastes. Now I have a nice email client that
intelligently downloads headers first, instead of downloading the entire
messages all in one shot. This lets me sort through and delete
any spam and viruses on the mail server, before wasting time downloading
them to my PC. (Thanks to my BadDNS milter I don't get much junk on this
email account, but some of my others are not as well filtered.) The
difference this makes when traveling, connecting over slow phone links
is just stupendous.
...Update
Of course by now you probably know that the Mozilla Suite was resurrected
as the all-volunteer SeaMonkey project. I'm glad to see that it still lives
on.
Mozilla Calendar
I've also been keeping the Mozilla Calendar extension alive in Seamonkey,
since the official Mozilla Calendar Project has abandoned it in favor
of the standalone Sunbird app and the Lightning extension. You can find
more on that here. -- 2007-03-01
In 1999 I went off with some colleagues and founded Symas Corporation to develop a directory-based identity and resource management system. This was a set of pluggable agents all inserted into an OpenLDAP slapd server. We developed this as a closed-source project while developing fixes and enhancements that would become part of the OpenLDAP 2.0 release. As time went on, we found that using the directory as an agent infrastructure was beyond most people's comprehension. It made investors' eyes glaze over as we explained the technology. We were too far ahead of the market.
In the meantime, we found that lots of people were struggling enough just to make LDAP behave as a static directory service. All of our expertise in working with OpenLDAP on our proprietary product turned out to have more wider application simply providing support for the basic services. This was a slight shift in focus for us, but a worthwhile one. We performed analyses leading into OpenLDAP 2.1 that allowed us to speed it up by over a factor of 200 compared the 2.0 code. And we completed development of the back-bdb transactional backend, a completely new animal for OpenLDAP providing robustness that the old back-ldbm backend simply couldn't offer. The 2.1 code also had a new "glue" concept that would allow pasting multiple database definitions into the appearance of a single directory tree. Yet another step toward integrating features that X.500 had long ago, and that common LDAP implementations had dropped.
OpenLDAP 2.2 saw yet another refactoring, yielding performance 50% faster than 2.1. It also saw the debut of the back-hdb hierarchical database, a concept that we first developed as a proprietary piece of Symas code back in 1999. (Indeed, most of the features we've added to OpenLDAP in the past few years were first used internally on Symas projects...) For the first time, OpenLDAP had a database that allowed directory subtrees to be renamed.
Also the glue mechanism was extended further, into the "overlay" concept which allows small modules to be written to customize the functionality of an existing backend. Until now, people looking for custom features would either have to (a) implement them in a scripting language (like perl), at a significant performance cost, (b) hack directly into an existing backend's code, at a significant maintenance cost, or (c) write their own backend from scratch, at a tremendous initial development cost. With overlays, one only needs to write as much code as the custom feature alone requires, and then the overlay can be configured on top of any backend without any further hassle. The feature set can be totally isolated from the main code base, making it far simpler to develop, debug, and maintain.
As 2005 rolls on we continue to plug away at enhancements for the OpenLDAP 2.3 release. There are more overlays, other new features, and a few more performance enhancements. The key feature from my perspective is the new config backend, which allows the static slapd.conf file to be migrated into its own LDAP database, that can be manipulated in realtime. No more restarting the server after manually editing a config file to make a change; just use ldapmodify and change the server setup on the fly. With this feature it will finally be practical to remotely administer the slapd servers from just about anywhere, using any graphical, web, or command line tool as you see fit, without any service interruption.
I should note that Hewlett-Packard has been sponsoring the majority of
the work we've put into OpenLDAP in the past year. Without them a lot of this
may never have happened. Indeed, the first version of back-config was
conceived in 2001 and never went anywhere because we didn't have the funds
to complete the work. It's nice to finally be able to get this online.
Thanks, HP. -- 2005-04-20
LDAP Performance
A lot has been published about OpenLDAP's (lack of) performance back in
2000. That was relevant for the 1.x releases and the 2.0 release family, which
were all essentially the original University of Michigan code with
minor bug fixes applied. Starting with the 2.1 release though, the
performance of OpenLDAP has been second to none. I presented some
new figures showing OpenLDAP's performance compared to some of the
other open source servers available out there at the
2007 SambaXP conference. In
short, our LDAP performance is many times faster than all the rest.
Here are my SambaXP slides presenting some of the data. I also discussed the results on the Connexitor Blog in a sequence of posts:
For example, here's a
Microsoft report on Active Directory vs Novell's Directory server, from 2000. This report
shows that AD is several times faster than Novell. A more recent Microsoft report
compares the LDAP performance of 64 bit ActiveDirectory vs 32 bit ActiveDirectory which
gives us some more context. From here we can estimate AD's position among the other servers
tested above. (Hint - with OpenLDAP at nearly 16,000 authentications (anonymous search + Simple Bind) per second,
and 32 bit ActiveDirectory at 3214 Simple Binds per second, and 64 bit AD at 3079 Simple Binds per second,
there's nothing new to tell here. OpenLDAP is the fastest directory software in the world, and nothing
else comes anywhere close. And oh yes, we don't cut any corners with schema validation or X.500 data model
compliance, unlike most of the others. Correctness always comes first.) Of course OpenLDAP doesn't just
deliver the fastest LDAP performance in the world. OpenLDAP also provides the most reliability, the most scalability,
the most flexibility, and the easiest manageability. By any measure, OpenLDAP is the best directory
software in the world, period.
-- 2007-05-02
Once upon a time, before Al Gore invented the Internet, I was the only Howard Chu in cyberspace. Now, to my dismay, I find that this name is far from unique, and people are confusing me with all the other Howard Chu's out there in the world. To attempt to clear up some of this confusion, here's a list of email addresses I've owned over the years...
(Photo by track photographer) |
2nd Gen Probe GT vs 3rd Gen RX-7 Unbelievable! V6 > 3R!! (Photos courtesy of Chris Ashton) |
Heavy-duty headlamp wiring for 2ndGen Ford Probes
Pictures of my car at night...
The window sticker from my Probe GTS.
SAE Paper Describing the design of the Probe's V6 engine. The document was scanned by Mike Paszti and the original 12 GIF files can be found at his site. I transcribed the document by hand and formatted it in HTML for easier viewing/printing. I also shrunk most of the diagrams to make them fit within an 800x600 window. Go to Mike's page if you need the full size images.
Fuel Rail pictures courtesy of Joe Kamman. Pictures of Nissan Altima fuel rails modified to fit the KL engine.
Me with an SR-71 "Blackbird" - the fastest production vehicle in the world. Awesome ride, terrible aftermarket support... (pic added 1999-09-20)