A Real-Time Weather Station

Before he became a famed digital privacy (and security) expert, Richard M. Smith ran a little software company called Phar Lap that specialized in DOS extenders. Back in October 1998, Dr. Dobb's Journal published his do-it-yourself article on creating your own weather station, highlighting the fun that could be had merging embedded devices with Internet technology.

A Real-Time Weather Station
by Richard M. Smith

One of the greatest difficulties facing embedded programmers is the lack of a user interface and user access to embedded devices. A couple of years ago, we realized that by leveraging existing Internet protocols and technology and merging them with embedded devices, we could create an Internet-ready embedded device that would be accessible to anyone using a web browser.

To demonstrate this technology, we've connected this web server to a commercially available weather instrumentation system that's located on the roof of our company offices in Cambridge, Massachusetts. Via a browser, you can obtain a current real-time report of the weather (see Figure 1) in Cambridge by accessing http://smallest.pharlap.com/. In addition to providing real-time weather data, this system could also be used for weather stations, medical instrumentation, factory-floor controllers, security systems, and smart-home devices.

Figure 1

The "World's Smallest Web Server"
To illustrate how you can use HTML as a universal GUI, we created a simple, yet flexible, Internet-based data logger called "Smallest." (We gave it this name because, to the best of our knowledge, it is perhaps the smallest web server on the Internet.) Smallest is an embedded system based on a PC/104 module stack (3.6x3.8x2.0 inches). It contains no hard disk and uses a flash-based file system (a flash disk) on the CPU module; see Figure 2.

Figure 2

Smallest sits in an office at Phar Lap, connected to a weather-collection module on the roof of the building via a serial cable. The weather-station module collects the current temperature, humidity, wind direction, and speed using the PC/104 systems, and the data is then made available to the Internet via the Web.

Smallest Hardware
As Figure 3 illustrates, the weather-station hardware is the Weather Monitor II from Davis Instruments (http://www.davisnet.com/). Mounted on the roof, this module communicates current weather data via an RS-232 serial cable connected to the COM1 port of our PC/104 system's CPU module. The system consists of a CPU module and an Ethernet LAN module from Real Time Devices (http://www.rtdusa.com/).

 Figure 3

A collection of PC/104 modules is often called a "stack" because the devices are actually stacked on top of each other via stackthrough connectors. There is no backplane in this system. The modules implement the backplane incrementally as more modules are added.

The CPU module is an i486 module that implements PC/AT-compatible architecture. To be PC/AT compatible, it must map standard I/O support circuitry, such as a direct memory access controller (DMAC) and programmable interrupt controller, into standard I/O locations. A PC/AT-compatible architecture also sports a PC-compatible BIOS, so it can boot from standard devices like floppy and hard disks. In our case, the BIOS also abstracts the flash memory on the CPU module as a hard-disk drive and allows the system to boot from it.
Equipped with an optional PC/104 VGA controller module, our CPU module could boot standard operating systems (such as Windows 95 or DOS), which run on PC/AT compatibles. However, since this device is an embedded data logger, we aren't very interested in such options.
Since we set out to build the smallest web server, we chose to implement the system using PC/104 modules. The system fits in a volume of approximately 4x4x2 inches. And since it requires only a 5-V power-supply voltage, we can use a compact power supply (at least compared to desktop systems). Some PC/104 CPU modules also support low-power features that make it possible to run the system from a battery with a solar panel.
PC/104 is also physically robust. Each board is secured on top of the next with four screws, and the stackthrough-header connectors present good electrical contact. Given their robust nature, PC/104 stacks have been built into applications such as avionics systems for aircraft and rockets. For example, Pegasus, a small commercial rocket that is launched from an airplane, uses PC/104. Of course, from our vantage point, the only advantage of a robust system is that it can survive an accidental plunge from the workbench.
PC/104 modules that are built to withstand extended environmental temperature and humidity ranges are available. This feature makes applications in hostile environments possible (factory floors or someone's garage or attic). In a nutshell, PC/104-based systems dare to go where desktop systems cannot.
Many modules are available in the PC/104 form factor: CPU modules from the 8088 to Pentium, DSP coprocessors, analog and digital I/O cards, as well as network adapters. To interface the data logger to the Internet, we selected the PC/104-based PCM-NE2000 Ethernet LAN card from WinSystems Inc. (http://www.winsystems.com/). Because it behaves much like an NE2000 ISA-bus card, this card enables us to connect the system to I/O devices commonly found on desktop-based PC/ATs. The Ethernet module is then connected to our company LAN, which connects to an ISP and the Internet through a standard router.
Smallest Server Software
To implement the data logger as an Internet-accessible web server, we need several software components. A TCP/IP stack is necessary for communication with other nodes on the Internet, the Ethernet device drivers, and a web server. In addition, we need a device driver for the weather-monitor hardware, which reads and logs the data. To tie it all together, we need an operating system with a footprint that is small enough to make it all work without a hard disk.
The TCP/IP stack refers to the software necessary to implement TCP/IP, which transports HTTP as well as most other Internet applications. For example, mail, which uses simple mail transfer protocol (SMTP), and ftp also use TCP/IP.
The most common TCP/IP API is the Socket API. Most operating systems implement this API or provide libraries to make their implementations compatible to it.
The protocol stack is also responsible for routing Internet packets between the local node and the external world through network adapters. Common network adapters are Ethernet cards, PPP interfaces through modems, and serial lines. A cellular digital packet data (CDPD) modem is a more exotic network interface.
Since we're using an Ethernet card as the network interface, we need a packet-level driver for it. This driver implements the bottom part of the TCP/IP stack and is responsible for framing and unframing IP packets transferred on an Ethernet LAN. Since we're using a common Ethernet network interface, finding a compatible driver shouldn't be a problem.
The web server for this application listens for HTTP connections from browsers and implements the GUI for our data logger. Besides sending HTML pages, it also needs to implement a forms-based interface so users can send information or parameters to the data logger via their browsers.

Now, we need an operating system to pull all of this together and write our application. For Smallest, we used Phar Lap's TNT Embedded ToolSuite (ETS), Realtime Edition. The ToolSuite's Realtime ETS Kernel is Phar Lap's Win32-compatible real-time operating system for the 32-bit PC/AT architecture.

The Realtime ETS Kernel also includes an Embedded Web Technology package, which provides all the components needed to set up an embedded web server using the ETS MicroWeb Server. The Realtime ETS Kernel also contains a WinSock API-compatible TCP/IP-stack implementation-ETS TCP/IP. Another interesting component is ETS HTML-On-The-Fly, a program-driven HTML generator.
With the Realtime ETS Kernel, we can use Windows-compatible DLLs for all the major components of our web-based data logger and program them using off-the-shelf 32-bit C compilers for Windows, such as Microsoft Visual C++ and Borland C++. Using DLLs enables a Realtime ETS Kernel-based system to be configured by simply plugging together DLL modules or plug-ins.
Once the kernel, ETSKERN.EXE, is loaded by the boot loader, it reads its configuration file to determine what else it needs to load to finish its configuration. Listing One is ETSKERN.INI, Smallest's kernel configuration file.

Listing One
[configuration]
console=local           ; Use the local console
filesystem=local        ; Use the local file system
configdir=\config       ; Where the .INI files live
bindir=\bin             ; Where the programs live

[logging]
logdir=\logfiles            ; Logging directory
logflags=0x0000_0003        ; Logging flags
logflush=no                 ; Don't flush the log file each write

[drivers]
eth-drv.drv                 ; Ethernet driver

[start]
microweb.dll

Configuration files are stored as ASCII text files on the system's flash disk and formatted as Windows .ini files. Using .ini files in an application is easy because the MicroWeb Server software provides library functions that read and parse the files.
Although DLLs can extend the kernel in many ways, we generally load three kinds of DLLs-I/O driver modules (*.DRV), API system-level modules, and HTML or application plug-ins. All DLLs are stored on the flash disk along with the kernel executable, configuration files, and HTML documents.
In Smallest, we have the choice of loading one of three Ethernet drivers, depending on the Ethernet adapter present. We're using an NE2000-compatible Ethernet card, so the Ethernet driver module ETH-NE2K.DRV must be copied to ETH-DRV.DRV, which is what the kernel will load. Although we currently support only one kind of weather-station hardware, the I/O driver that communicates with the weather station over the serial port is loaded as WSDRV.DRV. Other I/O-driver modules could be implemented and then simply configured by copying them to the flash disk from which Smallest boots.
Although the Realtime ETS Kernel implements a subset of the Win32 API by itself, it loads WSOCK32.DLL to add the WinSock API. The web server module loads from MICROWEB.DLL and implements the HTTP server and a library interface application that plug-ins can use for HTML-based interfaces.
The MicroWeb Server's initialization file controls which application modules and plug-ins are loaded. One of these, WEATHER.DLL, implements the weather interface and provides support for generating data and statistics for the various web pages.
Each plug-in links itself into the HTML name space. For example, when WEATHER.DLL loads, all references containing the path http://smallest.pharlap.com/weather/ cause a lookup to the WEATHER.DLL module to determine which function should be executed.
The HTML-On-The-Fly library can be linked with the embedded application, enabling routines in the application to generate HTML pages. This is analogous to programming with CGI scripts under UNIX, which provide roughly the same functionality-generating HTML documents at run time.
CGI scripts extend the web server by enabling you to implement custom functionality, like forms and dynamic HTML web pages. Under UNIX, CGI scripts are external to the web server and have to be executed from the file system. The server has set up the execution environment for the CGI script in such a way that the standard output, as seen by the script, is redirected to send HTML text to the client over the network.
Via HTML-On-The-Fly libraries in the kernel, the application can generate HTML dynamically and implement HTML forms. The program and library are already loaded in memory, so routines using HTML-On-The-Fly can execute without delay.

Furthermore, the HTML page to be sent is also constructed in memory before going out to the client, making HTML-On-The-Fly-based applications much more responsive than CGI-script-based applications. Also, HTML-On-The-Fly-based routines typically have much smaller run-time memory footprints than CGI scripts. These features are important when implementing HTML-based GUIs.

Listing Two illustrates an HTML-On-The-Fly plug-in that shows the date, time, and number of times it has been accessed. HTML-On-The-Fly also enables a program to extract form data sent by a web browser. While this example may seem trivial, we can use the same technique to implement complete GUIs. In fact, plug-ins such as ADMIN.DLL and SECURITY.DLL use this technique.


Another method for generating HTML pages with data is via server-side HTML. Server-side HTML uses keywords in the contents of an HTML document that is stored on the flash disk to key the plug-in to generate data and substitute it for the keys.
Listing Three is a server-side HTML document, which is a standard HTML document with field keys embedded in it. These fields are filled in when the document is presented by the ETS MicroWeb server. These keys have a general format of . For example, the current time is substituted for the field . 

 

This scheme of using fields in standard HTML documents makes it easy for users to customize products by creating their own HTML documents from standard authoring tools like FrontPage. This allows a graphic artist to design the page based on presentation without having to know the ins-and-outs of programming a plug-in in C and HTML-On-The-Fly. The server-side HTML document is simply stored on the device's flash disk as an HTML document.

The Ethernet driver module, ETH-DRV.DLL, also has a configuration file (as in Listing Four) because it needs to know which IP address to bind with the network interface. This is programmed via the IPADDR field. Of course, the net mask is also programmed here.
Listing Four
[eth-ne2k]
port=auto                      ; I/O port number (or auto)
irq=11                         ; IRQ (or auto)
ethernetaddr=00:00:00:00:00:00 ; 6 byte Ethernet address
ipaddr=192.107.36.250          ; IP address (or bootp or rarp)
subnetmask=255.255.255.0       ; Subnet mask (or bootp)


Once the Ethernet module is loaded and configured, we can look at the TCP/IP stack. The WinSock-compatible TCP/IP module needs to know where to find the name server and the gateway (for instance, the router sending the IP packets to the outside world). Listing Five is the configuration file for the WinSock module.

Listing Five
[Winsock]
numsockets=50                  ; Number of active TCP sockets
gatewayaddr=192.107.36.11      ; Gateway IP address
bootp=no                       ; Use Bootp protocol for IP addresses
dnsserveraddr=192.107.36.9     ; DNS server IP address
dnsretries=default             ; Number of DNS retries
dnsretrywait=default           ; Wait time for DNS retry


These are all the critical configurations that Smallest needs to boot up and start operating. The MicroWeb Server now needs to know which plug-ins to load in order to serve web pages on the Internet. This is controlled from the configuration file for MICROWEB.DLL (as in Listing Six), which lists all the plug-ins.

Listing Six
[MicroWeb]
plugindir=\bin      ; Where the plugins live
htmldir=\html       ; HTML directory

[plugins]
admin=admin         ; Web server admin
fcache=fcache       ; File Cacher
security=security   ; Security plugin
ftpserv=ftp         ; FTP server
onldebug=debug      ; Online debugger
kybint3=debug       ; INT3 breakpoint from keyboard
fingserv=finger     ; Finger server
ping=ping           ; Ping
tcpip=tcpip         ; TCP/IP
htmltest=htmltest   ; HTML test
intbench=bench      ; Benchmark plugin
wsdrv.drv=wsdrv     ; Davis weather station I/O driver
weather=weather     ; Internet Weather Station software

[Website info]
host_name=      "Unknown"
title=          "MicroWeb Server"
organization=   "Phar Lap Software, Inc."
address1=       "60 Aberdeen Ave."
address2=
city=           "Cambridge"
state=          "MA"
zip=            "02138"
country=        "USA"
phone=          "(617) 661-1510"
fax=            "(617) 876-2972"
webmaster_email=    "[email protected]"
info_email=     "[email protected]"
www_url=        "http://www.pharlap.com"
ftp_url=        "ftp://ftp.pharlap.com"
computer_model= "PC Compatible"
stock_ticker_symbol=
copyright=      "(C) 1997 Phar Lap Software, Inc."
last_updated=   "Feb. 11, 1997"
longitude=      "71 07 37 W"
latitude=       "42 23 58 N"

[formats]
units=English       ; English units
timeformat=12ampm   ; 12 hour AM/PM time format

[Mime Types]
 .avi=video/avi
 .doc=application/msword
 .gif=image/gif
 .htm=text/html
 .html=text/html
 .jpg=image/jpeg
 .ps=application/postscript
 .rtf=application/rtf
 .tif=image/tiff
 .txt=text/plain
 .wav=audio/wav
 .zip=application/zip
 .ocx=application/x-oleobject

Once Smallest is running, we can customize the web-server presentation by changing the appropriate field in the configuration file and the underlying HTML pages on the flash disk.
Phar Lap offers the collection of software that composes Smallest on a single disk called the Internet Weather Station software (IWS). With this kit, you can turn any PC/AT-compatible system with a serial port and NE2000-compatible network adapter into a weather station by interfacing to the Davis Weather Monitor II. The kit also includes drivers for 3COM- and SMC-based network adapters and supports PCMCIA-based network adapters that are compatible with any of these types.
Serving Your Own
There are a range of embedded-system applications that can be built to be web accessible. In addition to the smart data logger described here, for instance, this technology is well-suited for medical instrumentation (such as in-room patient monitors), transportation systems (global positioning systems), factory-floor controllers, security systems, office machines, data-acquisition systems, and more. Essentially, the technology can be used for any distributed and highly configurable or customizable sensor network that can be accessed or configured with a web browser by using HTML. And because all of this technology is based on freely available protocols, it is possible to design such an application from scratch.