2 % This document is released under the GPL
3 % Initially written by Stefan Reinauer, <stepan@openbios.org>
6 \documentclass[titlepage,12pt]{article}
10 \usepackage[pdftex]{hyperref}
11 % \usepackage{makeidx}
15 urlbordercolor={1 1 1},
16 menubordercolor={1 1 1},
17 linkbordercolor={1 1 1},
19 % pdfpagemode=None, % PDF-Viewer starts without TOC
21 pdftitle={LinuxBIOS on AMD64},
22 pdfauthor={Stefan Reinauer},
23 pdfsubject={LinuxBIOS configuration and build process},
24 pdfkeywords={LinuxBIOS, Opteron, AMD64, Athlon64, Build}
28 % \newcommand{\sh}[1]{\begin{verbatim}\texttt{#1}\end{verbatim}}
29 % \newcommand{\prog}[1]{\textit{#1}}
31 \setlength{\parindent}{0pt}
33 \title{LinuxBIOS on AMD64}
34 \author{Stefan Reinauer $<$stepan@openbios.org$>$}
53 This document targets porting LinuxBIOS to new mainboards and creating
54 custom firmware images using LinuxBIOS. It describes how to build
55 LinuxBIOS images for the AMD64 platform, including hypertransport
56 configuration and pertinent utilities. If you are missing information or
57 find errors in the following descriptions, contact
58 \href{mailto:stepan@openbios.org}{\textit{Stefan Reinauer $<$stepan@openbios.org$>$}}
67 \item 2004/06/02 url and language fixes from Ken Fuchs $<$kfuchs@winternet.com$>$
68 \item 2004/02/10 acpi and option rom updates
69 \item 2003/11/18 initial release
78 \section{What is LinuxBIOS?}
80 LinuxBIOS aims to replace the normal BIOS found on PCs, Alphas, and
81 other machines with a Linux kernel that can boot Linux from a cold
82 start. The startup code of an average LinuxBIOS port is about 500 lines
83 of assembly and 5000 lines of C. It executes 16 instructions to get into
84 32bit mode and then performs DRAM and other hardware initializations
85 required before Linux can take over.
87 The projects primary motivation initially was maintenance of large
88 clusters. Not surprisingly interest and contributions have come from
89 people with varying backgrounds. Nowadays a large and growing number of
90 Systems can be booted with LinuxBIOS, including embedded systems,
91 Desktop PCs and Servers.
94 % 4 Build Requirements
97 \section{Build Requirements}
98 To build LinuxBIOS for AMD64 from the sources you need a recent Linux
99 system for x86 or AMD64. SUSE Linux 8.2 or 9.0 are known to work fine.
100 The following toolchain is recommended:
104 \item binutils 2.14.90.0.5
109 \textbf{NOTE:} Later versions should also work. Prior versions might lead to problems.
114 % 5 Getting the Sources
117 \section{Getting the Sources}
119 The latest LinuxBIOS sources are available via CVS. The CVS repository
120 is maintained at SourceForge.net (the project name is \emph{FreeBIOS}).
121 First you should create a directory for your LinuxBIOS trees:
130 You can get the entire source tree via CVS:
134 $ cvs -d:pserver:anonymous@cvs.sourceforge.net:/cvsroot/freebios login
138 Hit return when you are asked for a password. Then checkout (or update)
139 the freebios source tree as follows:
143 $ cvs -z3 -d:pserver:anonymous@cvs.sourceforge.net:/cvsroot/freebios co freebios2
147 Once the source tree is checked out, it can be updated with:
155 Due to recent problems with SourceForge's CVS infrastructure we set up a
156 snapshot site that keeps hourly source trees of the last four days. It
157 is available at \url{http://snapshots.linuxbios.org/}.
158 Due to major structural enhancements to \hbox{LinuxBIOS}, AMD64 support
159 is only available in the \texttt{freebios2} tree. This tree reflects (as
160 of November 2003) LinuxBIOS version 1.1.5 and will lead to LinuxBIOS 2.0
161 when finished. Most x86 hardware is currently only supported by the
165 % 6 LinuxBIOS configuration overview
168 \section{LinuxBIOS configuration overview}
169 To support a large variety of existing hardware LinuxBIOS allows for a
170 lot of configuration options that can be tweaked in several ways:
174 Firmware image specific configuration options can be set in the image
175 configuration file which is usually found in
176 \texttt{freebios2/targets/$<$vendor$>$/$<$mainboard$>$/}. Such
177 options are the default amount of output verbosity during bootup, image
178 size, use of fallback mechanisms, firmware image size and payloads
179 (Linux Kernel, Bootloader...) The default configuration file name is
180 \texttt{Config.lb}, but LinuxBIOS allows multiple configurations to
181 reside in that directory.
183 \item Mainboard specific configuration options can be set in the
184 mainboard configuration file placed in
185 \texttt{freebios2/src/mainboard/$<$vendor$>$/$<$mainboard$>$}. The
186 mainboard configuration file is always called \texttt{Config.lb}. It
187 contains information on the onboard components of the mainboard like
188 CPU type, northbridge, southbridge, hypertransport configuration and
189 SuperIO configuration. This configuration file also allows to include
190 addon code to hook into the LinuxBIOS initialization mechanism at
195 This document describes different approaches of changing and configuring the
196 LinuxBIOS source tree when building for AMD64.
199 % 7 Building LinuxBIOS
202 \section{Building LinuxBIOS}
203 One of the design goals for building LinuxBIOS was to keep object files
204 out of the source tree in a separate place. This is mandatory for
205 building parallel LinuxBIOS images for several distinct mainboards
206 and/or platforms. Therefore building LinuxBIOS consists of two steps:
209 creating a build tree which holds all files automatically created by the
210 configuration utility and the object files
212 compiling the LinuxBIOS code and creating a flashable firmware image.
215 The first of these two steps is accomplished by the \texttt{buildtarget}
216 script found in \texttt{freebios2/targets/}. To build LinuxBIOS for
217 instance for the AMD Solo Athlon64 mainboard enter:
220 % cd freebios2/targets
221 % ./buildtarget amd/solo
224 This will create a directory containing a Makefile and other software
225 components needed for this build. The directory name is defined in the
226 firmware image specific configuration file. In the case of AMD's Solo
227 mainboard the default directory resides in
228 \texttt{freebios2/targets/amd/solo/solo}. To build the LinuxBIOS image, do
235 The LinuxBIOS image filename is specified in the firmware image specific
236 configuration file. The default filename for AMD's Solo mainboard is
240 % 8 Programming LinuxBIOS to flash memory
243 \section{Programming LinuxBIOS to flash memory}
244 The image resulting from a LinuxBIOS build can be directly programmed to
245 a flash device, either using a hardware flash programmer or by using the
246 Linux flash driver devbios or mtd. This document assumes that you use a
247 hardware flash programmer. If you are interested in doing in-system
248 software flash programming, find detailed information:
251 \item \url{http://www.openbios.org/development/devbios.html} (/dev/bios)
252 \item \url{http://www.linux-mtd.infradead.org/} (Memory Technology Device Subsystem MTD)
258 % 9 LinuxBIOS configuration
261 \section{LinuxBIOS configuration}
262 The following chapters will cope with configuring LinuxBIOS. All
263 configuration files share some basic rules
266 The default configuration file name in LinuxBIOS is \texttt{Config.lb}.
268 All variables used in a configuration file have to be declared in this
269 file with \texttt{uses VARNAME} before usage.
271 Comments can be added on a new line by using the comment identifier
272 \texttt{\#} at the beginning of the line.
274 LinuxBIOS distinguishes between statements and options. Statements cause
275 the LinuxBIOS configuration mechanism to act, whereas options set
276 variables that are used by the build scripts or source code.
278 Default configuration values can be set in the mainboard configuration
279 files (keyword default)
281 Option overrides to the default configuration can only be specified in
282 the build target configuration file
283 \texttt{freebios2/targets/$<$vendor$>$/$<$mainboard$>$/Config.lb}
287 \subsection{Common Configuration Statements}
291 \item \begin{verbatim}uses\end{verbatim}
293 All local configuration variables have to be declared before they can be
299 \textbf{NOTE:} Only configuration variables known to the configuration
300 system can be used in configuration files. LinuxBIOS checks
301 \texttt{freebios2/src/config/Options.lb} to see whether a configuration
304 \item \begin{verbatim}default\end{verbatim}
306 The \texttt{default} statement is used to set a configuration variable
307 with an overridable default value. It is commonly used in mainboard
313 default ROM_IMAGE_SIZE=0x10000
316 It is also possible to assign the value of one configuration variable to
320 default FALLBACK_SIZE=ROM_SIZE
323 Also, simple expressions are allowed:
326 default FALLBACK_SIZE=(ROM_SIZE - NORMAL_SIZE)
329 If an option contains a string, this string has to be protected with
333 default CC="gcc m32"
336 \item \begin{verbatim}option\end{verbatim}
338 The \texttt{option} statement basically behaves identically to the
339 \texttt{default} statement. But unlike default it can only be used in
340 build target configuration files
341 (\texttt{freebios2/targets/$<$vendor$>$/$<$mainboard$>$}). The option
342 statement allows either to set new options or to override default values
343 set with the default statement in a mainboard configuration file.
344 Syntax and application are the same as with default.
348 \subsection{Firmware image specific configuration}
349 LinuxBIOS allows to create different firmware images for the same
350 hardware. Such images can differ in the amount of output they produce,
351 the payload, the number of subimages they consist of etc.
353 The firmware image specific configuration file can be found in \\
354 \texttt{freebios2/targets/$<$vendor$>$/<mainboard$>$}.
356 \subsubsection{Firmware image specific keywords}
357 In addition to the above described keywords the following statements are
358 available in firmware image specific configuration files:
361 \item \begin{verbatim}romimage\end{verbatim}
363 The \texttt{romimage} definition describes a single rom build within the
364 final LinuxBIOS image. Normally there are two romimage definitions per
365 LinuxBIOS build: \texttt{normal} and \texttt{fallback}.
367 Each \texttt{romimage} section needs to specify a mainboard directory and a
368 payload. The mainboard directory contains the mainboard specific
369 configuration file and source code. It is specified relatively to
370 \texttt{freebios2/src/mainboard}. The payload definition is an absolute
371 path to a static elf binary (i.e Linux kernel or etherboot)
375 option USE_FALLBACK_IMAGE=0
376 option ROM_IMAGE_SIZE=0x10000
377 option LINUXBIOS_EXTRA_VERSION=".0Normal"
379 payload /suse/stepan/tg3ide_
384 \item \begin{verbatim}buildrom\end{verbatim}
386 The \texttt{buildrom} statement is used to determine which of the
387 LinuxBIOS image builds (created using \texttt{romimage}) are packed
388 together to the final LinuxBIOS image. It also specifies the order of
389 the images and the final image size:
392 buildrom ./solo.rom ROM_SIZE "normal" "fallback"
398 \subsubsection{Firmware image configuration options}
399 In addition to the definitions described above there are a number of
400 commonly used options. Configuration options set in the firmware image
401 specific configuration file can override default selections from the
402 Mainboard specific configuration. See above examples about
403 option on how to set them.
407 \item \begin{verbatim}CC\end{verbatim}
409 Target C Compiler. Default is \texttt{\$(CROSS\_COMPILE)gcc}. Set to
410 \texttt{gcc m32} for compiling AMD64 LinuxBIOS images on an AMD64
413 \item \begin{verbatim}CONFIG_CHIP_CONFIGURE \end{verbatim}
415 Use new \textit{chip\_configure} method for configuring (nonpci)
416 devices. Set to \texttt{1} for all AMD64 mainboards.
418 \item \begin{verbatim}MAXIMUM_CONSOLE_LOGLEVEL\end{verbatim}
420 Errors or log messages up to this level can be printed. Default is
421 \texttt{8}, minimum is \texttt{0}, maximum is \texttt{10}.
423 \item \begin{verbatim}DEFAULT_CONSOLE_LOGLEVEL\end{verbatim}
425 Console will log at this level unless changed. Default is \texttt{7},
426 minimum is \texttt{0}, maximum is \texttt{10}.
428 \item \begin{verbatim}CONFIG_CONSOLE_SERIAL8250\end{verbatim}
430 Log messages to 8250 uart based serial console. Default is \texttt{0}
431 (don't log to serial console). This value should be set to \texttt{1}
432 for all AMD64 builds.
434 \item \begin{verbatim}ROM_SIZE\end{verbatim}
436 Size of final ROM image. This option has no default value.
438 \item \begin{verbatim}FALLBACK_SIZE\end{verbatim}
440 Fallback image size. Defaults to \texttt{65536} bytes. \textbf{NOTE:}
441 This does not include the fallback payload.
443 \item \begin{verbatim}HAVE_OPTION_TABLE\end{verbatim}
445 Export CMOS option table. Default is \texttt{0}. Set to \texttt{1} if
446 your mainboard has CMOS memory and you want to use it to store
447 LinuxBIOS parameters (Loglevel, serial line speed, ...)
449 \item \begin{verbatim}CONFIG_ROM_STREAM\end{verbatim}
451 Boot image is located in ROM (as opposed to \texttt{CONFIG\_IDE\_STREAM}, which
452 will boot from an IDE disk)
454 \item \begin{verbatim}HAVE_FALLBACK_BOOT\end{verbatim}
456 Set to \texttt{1} if fallback booting is required. Defaults to
462 The following options should be used within a romimage section:
466 \item \begin{verbatim}USE_FALLBACK_IMAGE\end{verbatim}
468 Set to \texttt{1} to build a fallback image. Defaults to \texttt{0}
470 \item \begin{verbatim}ROM_IMAGE_SIZE\end{verbatim}
472 Default image size. Defaults to \texttt{65535} bytes.
474 \item \begin{verbatim}LINUXBIOS_EXTRA_VERSION\end{verbatim}
476 LinuxBIOS extra version. This option has an empty string as default. Set
477 to any string to add an extra version string to your LinuxBIOS build.
483 \subsection{Mainboard specific configuration}
485 Mainboard specific configuration files describe the onboard
486 mainboard components, i.e. bridges, number and type of CPUs. They also
487 contain rules for building the low level start code which is translated
488 using romcc and/or the GNU assembler. This code enables caches and
489 registers, early mtrr settings, fallback mechanisms, dram init and
492 \textbf{NOTE:} The \texttt{option} keyword can not be used in mainboard
493 specific configuration files. Options shall instead be set using the
494 \texttt{default} keyword so that they can be overridden by the image
495 specific configuration files if needed.
497 \subsubsection{Mainboard specific keywords}
499 The following statements are used in mainboard specific configuration
503 \item \begin{verbatim}arch\end{verbatim}
505 Sets the CPU architecture. This should be \texttt{i386} for AMD64 boards.\\
512 \item \begin{verbatim}cpu\end{verbatim}
514 The cpu statement is needed once per possibly available CPU. In a
515 one-node system, write:
521 \item \begin{verbatim}driver\end{verbatim}
523 The \texttt{driver} keyword adds an object file to the driver section of a
524 LinuxBIOS image. This means it can be used by the PCI device
525 initialization code. Example:
531 \item \begin{verbatim}object\end{verbatim}
533 The \texttt{object} keyword adds an object file to the LinuxBIOS image.
534 Per default the object file will be compiled from a \texttt{.c} file
535 with the same name. Symbols defined in such an object file can be used
536 in other object files and drivers. Example:
542 \item \begin{verbatim}makerule\end{verbatim}
544 This keyword can be used to extend the existing file creation rules
545 during the build process. This is useful if external utilities have to
546 be used for the build. LinuxBIOS on AMD64 uses romcc for it's early
547 startup code placed in auto.c.
549 To tell the configuration mechanism how to build \texttt{romcc} files,
554 depends "$(MAINBOARD)/auto.c"
555 action "$(CPP) I$(TOP)/src $(ROMCCPPFLAGS) $(CPPFLAGS) \
556 $(MAINBOARD)/auto.c > ./auto.E"
559 depends "./auto.E ./romcc"
560 action "./romcc mcpu=k8 O ./auto.E > auto.inc"
564 Each \texttt{makerule} section contains file dependencies (using the
565 texttt{depends} keyword) and an action that is taken when the dependencies
566 are satisfied (using the \texttt{action} keyword).
568 \item \begin{verbatim}mainboardinit\end{verbatim}
570 With the mainboardinit keyword it's possible to include assembler code
571 directly into the LinuxBIOS image. This is used for early infrastructure
572 initialization, i.e. to switch to protected mode. Example:
575 mainboardinit cpu/i386/entry16.inc
578 \item \begin{verbatim}ldscript\end{verbatim}
580 The GNU linker ld is used to link object files together to a LinuxBIOS
583 Since it is a lot more comfortable and flexible to use the GNU linker
584 with linker scripts (ldscripts) than to create complex command line
585 calls, LinuxBIOS features including linker scripts to control image
589 ldscript /cpu/i386/entry16.lds
593 \item \begin{verbatim}dir\end{verbatim}
595 LinuxBIOS reuses as much code between the different ports as possible.
596 To achieve this, commonly used code can be stored in a separate
597 directory. For a new mainboard, it is enough to know that the code in
598 that directory can be used as is.
600 LinuxBIOS will also read a \texttt{Config.lb} file stored in that
601 directory. This happens with:
608 \item \begin{verbatim}config\end{verbatim}
610 This keyword is needed by the new chip configuration scheme. Should be
617 \item \begin{verbatim}register\end{verbatim}
618 The \texttt{register} keyword can occur in any section, passing
620 parameters to the code handling the associated device.
624 register "com1" = "{1, 0, 0x3f8, 4}"
627 \item \begin{verbatim}northbridge\end{verbatim}
629 The \texttt{northbridge} keyword describes a system northbridge. Some
630 systems, like AMD64, can have more than one northbridge, i.e. one per
631 CPU node. Each northbridge is described by the path to the northbridge
632 code in LinuxBIOS (relative to \texttt{freebios2/src/northbridge}), i.e.
633 \texttt{amd/amdk8} and a unique name (i.e \texttt{mc0}) \\
637 northbridge amd/amdk8 "mc0"
642 \item \begin{verbatim}southbridge\end{verbatim}
644 To simplify the handling of bus bridges in a LinuxBIOS system, all
645 bridges available in a system that are not northbridges (i.e AGP, IO,
646 PCIX) are seen as southbridges.
648 Since from the CPUs point of view any southbridge is connected via the
649 northbridge, a southbridge section is declared within the northbridge
650 section of the north bridge it is attached to.
652 Like the northbridge, any other bridge is described by the path to it's
653 driver code, and a unique name. If the described bridge is a
654 hypertransport device, the northbridge's hypertransport link it connects
655 to can be specified using the \texttt{link} keyword. Example:
658 northbridge amd/amdk8 "mc0"
660 southbridge amd/amd8111 "amd8111" link 0
667 \item \begin{verbatim}pci\end{verbatim}
669 The \texttt{pci} keyword can only occur within a \texttt{northbridge} or
670 \texttt{southbridge} section. It is used to describe the PCI devices
671 that are provided by the bridge. Generally all bridge sections have a
672 couple of \texttt{pci} keywords.
674 The first occurrence of the \texttt{pci} keyword tells LinuxBIOS where
675 the bridge devices start, relative to the PCI configuration space used
676 by the bridge. The following occurences of the \texttt{pci} keyword
677 describe the provided devices.
679 Adding the option \texttt{on} or \texttt{off} to a PCI device will
680 enable or disable this device. This feature can be used if some bridge
681 devices are not wired to hardware outputs and thus are not used.
686 northbridge amd/amdk8 "mc1"
699 southbridge amd/amd8111 "amd8111" link 0
715 \item \begin{verbatim}superio\end{verbatim}
717 SuperIO devices are basically handled like brigdes. They are taking
718 their driver code from \texttt{freebios2/src/superio/}. They don't
719 provide a PCI compatible configuration interface, but instead are ISA
720 PnP devices. Normally they are connected to a southbridge. If this is
721 the case, the \texttt{superio} section will be a subsection of the
722 \texttt{southbridge} section of the southbridge it is connected to.
726 superio NSC/pc87360 link 1
738 register "com1" = "{1, 0, 0x3f8, 4}"
739 register "lpt" = "{1}"
747 \subsubsection{Mainboard specific configuration options}
749 The following options are commonly used in mainboard specific
752 They should be set using the \texttt{default} keyword:
756 \item \begin{verbatim}HAVE_HARD_RESET\end{verbatim}
758 If set to \texttt{1}, this option defines that there is a hard reset
759 function for this mainboard. This option is not defined per default.
761 \item \begin{verbatim}HAVE_PIRQ_TABLE\end{verbatim}
763 If set to \texttt{1}, this option defines that there is an IRQ Table for
764 this mainboard. This option is not defined per default.
766 \item \begin{verbatim}IRQ_SLOT_COUNT\end{verbatim}
768 Number of IRQ slots. This option is not defined per default.
770 \item \begin{verbatim}HAVE_MP_TABLE\end{verbatim}
772 Define this option to build an MP table (v1.4). The default is not to
775 \item \begin{verbatim}HAVE_OPTION_TABLE\end{verbatim}
777 Define this option to export a CMOS option table. The default is not to
778 export a CMOS option table.
780 \item \begin{verbatim}CONFIG_SMP\end{verbatim}
782 Set this option to \texttt{1} if the mainboard supports symmetric
783 multiprocessing (SMP). This option defaults to \texttt{0} (no SMP).
785 \item \begin{verbatim}CONFIG_MAX_CPUS\end{verbatim}
787 If \begin{verbatim}CONFIG_SMP\end{verbatim} is set, this option defines
788 the maximum number of CPUs (i.e. the number of CPU sockets) in the
789 system. Defaults to \texttt{1}.
791 \item \begin{verbatim}CONFIG_IOAPIC\end{verbatim}
793 Set this option to \texttt{1} to enable IOAPIC support. This is
794 mandatory if you want to boot a 64bit Linux kernel on an AMD64 system.
796 \item \begin{verbatim}STACK_SIZE\end{verbatim}
798 LinuxBIOS stack size. The size of the function call stack defaults to
799 \texttt{0x2000} (8k).
801 \item \begin{verbatim}HEAP_SIZE\end{verbatim}
803 LinuxBIOS heap size. The heap is used when LinuxBIOS allocates memory
804 with malloc(). The default heap size is \texttt{0x2000}, but AMD64 boards
805 generally set it to \texttt{0x4000} (16k)
807 \item \begin{verbatim}XIP_ROM_BASE\end{verbatim}
809 Start address of area to cache during LinuxBIOS execution directly from
812 \item \begin{verbatim}XIP_ROM_SIZE\end{verbatim}
814 Size of area to cache during LinuxBIOS execution directly from ROM
816 \item \begin{verbatim}CONFIG_COMPRESS\end{verbatim}
818 Set this option to \texttt{1} for a compressed image. If set to
819 \texttt{0}, the image is bigger but will start slightly faster (since no
820 decompression is needed).
828 % 10. Tweaking the source code
831 \section{Tweaking the source code}
832 Besides configuring the existing code it is sometimes necessary or
833 desirable to tweak certain parts of LinuxBIOS by direct changes to the
834 code. This chapter covers some possible enhancements and changes that
835 are needed when porting LinuxBIOS to a new mainboard or just come
838 \subsection{Hypertransport configuration}
839 Before LinuxBIOS is able to activate all CPUs and detect bridges
840 attached to these CPUs (and thus, see all devices attached to the
841 system) it has to initialize the coherent hypertransport devices.
843 The current algorithms to do coherent hypertransport initialization are
844 not fully, automatically evaluating the hypertransport chain graph.
845 Therefore the code needs to be adapted when porting LinuxBIOS to a new
846 AMD64 mainboard. An example arrangement of hypertransport devices
851 \includegraphics[scale=1.0]{hypertransport.pdf}
852 \caption{Example: Hypertransport Link Connections}
853 \label{fix:hypertransport}
856 Each hypertransport device has one to three hypertransport links that
857 are used for device interconnection. These links are called LDT$[$012$]$, or
858 accordingly UP, ACROSS, DOWN. Communication between the hypertransport
859 devices can be freely routed, honoring the physical wiring. Teaching the
860 coherent hypertransport initialization algorithm this wiring happens in
866 \item Setting outgoing connections
867 The algorithm needs to know which outgoing port of a CPU node is
868 connected to the directly succeeding node. This is done in
869 \texttt{freebios2/src/mainboard/$<$vendor$>$/$<$mainboard$>$/auto.c}
870 with a number of preprocessor defines (one define for two-node systems,
871 three defines for four-node systems).
873 The ports in question are flagged with a circle in the graph for
874 illustration. For the example graph above (all outgoing connections are
875 realized using LDT1/ACROSS) the defines are:
878 #define CONNECTION_0_1 ACROSS
879 #define CONNECTION_0_2 ACROSS
880 #define CONNECTION_1_3 ACROSS
883 \item Describing routing information between CPUs.
885 There are basically three different message types for hypertransport
888 \item request packages
889 \item response packages
890 \item broadcast packages
893 These three message types are routed using different hypertransport
894 ports. The routing information is written to the AMD K8 routing table.
895 In an Nnode system this routing table consists of 3*N*N entries , one
896 for each message type and for each possible CPU --> CPU communication. For
897 simplicity LinuxBIOS keeps the 3 routing entries for each CPU --> CPU
898 communication in one machine word. The routing table of each node looks
902 /* Routing Table for Node i
904 * F0: 0x40, 0x44, 0x48, 0x4c, 0x50, 0x54, 0x58, 0x5c
905 * i: 0, 1, 2, 3, 4, 5, 6, 7
907 * [ 0: 3] Request Route
908 * [0] Route to this node
909 * [1] Route to Link 0
910 * [2] Route to Link 1
911 * [3] Route to Link 2
912 * [11: 8] Response Route
913 * [0] Route to this node
914 * [1] Route to Link 0
915 * [2] Route to Link 1
916 * [3] Route to Link 2
917 * [19:16] Broadcast route
918 * [0] Route to this node
919 * [1] Route to Link 0
920 * [2] Route to Link 1
921 * [3] Route to Link 2
925 The routing table is passed to the coherent hypertransport
926 initialization algorithm by defining a function called
927 \texttt{generate\_row()} in \texttt{auto.c}:
930 static unsigned int generate_row
931 (uint8_t node, uint8_t row, uint8_t maxnodes)
934 This function is trivial if there is only one CPU in the system, since
935 no routing has to be done:
938 static unsigned int generate_row
939 (uint8_t node, uint8_t row, uint8_t maxnodes)
941 return 0x00010101; /* default row entry */
945 On a two-node system things look slightly more complicated. Since the
946 coherent hypertransport initialization algorithm works by consecutively
947 enabling CPUs, it contains routing information for driving the system
948 with one node and two nodes:
951 static unsigned int generate_row
952 (uint8_t node, uint8_t row, uint8_t maxnodes)
954 uint32_t ret=0x00010101; /* default row entry */
955 static const unsigned int rows_2p[2][2] = {
956 { 0x00050101, 0x00010404 },
957 { 0x00010404, 0x00050101 }
959 if(maxnodes>2) maxnodes=2;
960 if (!(node>=maxnodes || row>=maxnodes)) {
961 ret=rows_2p[node][row];
967 Systems with four nodes have to contain routing information for one, two
968 and four-node setups:
971 static unsigned int generate_row
972 (uint8_t node, uint8_t row, uint8_t maxnodes)
974 uint32_t ret=0x00010101; /* default row entry */
975 static const unsigned int rows_2p[2][2] = {
976 { 0x00030101, 0x00010202 },
977 { 0x00010202, 0x00030101 }
979 static const unsigned int rows_4p[4][4] = {
980 { 0x00070101, 0x00010202, 0x00030404, 0x00010204 },
981 { 0x00010202, 0x000b0101, 0x00010208, 0x00030808 },
982 { 0x00030808, 0x00010208, 0x000b0101, 0x00010202 },
983 { 0x00010204, 0x00030404, 0x00010202, 0x00070101 }
985 if (!(node>=maxnodes || row>=maxnodes)) {
987 ret=rows_2p[node][row];
989 ret=rows_4p[node][row];
996 \subsection{DRAM configuration}
997 Setting up the RAM controller(s) is probably the most complex part of
998 LinuxBIOS. Basically LinuxBIOS serially initializes all RAM controllers
999 in the system, using SPDROM (serial presence detect) data to set
1000 timings, size and other properties. The SPD data is usually read
1001 utilizing the I2C SMBUS interface of the southbridge.
1003 There is one central data structure that describes the RAM controllers
1004 available on an AMD64 system and the associated devices:
1007 struct mem_controller {
1009 device_t f0, f1, f2, f3;
1010 uint8_t channel0[4];
1011 uint8_t channel1[4];
1015 Available mainboard implementations and CPUs create the need to add
1016 special setup code to RAM initialization in a number of places.
1017 LinuxBIOS provides hooks to easily add code in these places without
1018 having to change the generic code. Whether these hooks have to be used
1019 depends on the mainboard design. In many cases the functions executed
1020 by the hooks just carry out trivial default settings or they are even
1023 Some mainboard/CPU combinations need to trigger an additional memory
1024 controller reset before the memory can be initialized properly. This is,
1025 for example, used to get memory working on preC stepping AMD64
1026 processors. LinuxBIOS provides two hooks for triggering onboard memory
1030 \item \begin{verbatim}static void memreset_setup(void)\end{verbatim}
1031 \item \begin{verbatim}static void memreset(int controllers, const struct
1032 mem_controller *ctrl)\end{verbatim}
1035 Some mainboards utilize an SMBUS hub or possibly other mechanisms to
1036 allow using a large number of SPDROMs and thus ram sockets. The result
1037 is that only the SPDROM information of one cpu node is visible at a
1038 time. The following function, defined in \texttt{auto.c}, is called every time
1039 before a memory controller is initialized and gets the memory controller
1040 information of the next controller as a parameter:
1043 static inline void activate_spd_rom (const struct mem_controller *ctrl)
1046 The way SMBUS hub information is coded into the \texttt{mem\_controller}
1047 structure is mainboard implementation specific and not
1048 described here. See \texttt{freebios2/src/mainboard/amd/quartet/auto.c}
1051 LinuxBIOS folks have agreed on SPD data being the central information
1052 source for RAM specific information. But not all mainboards/RAM
1053 modules feature a physical SPD ROM. To still allow an easy to use SPD
1054 driven setup, there is a hook that abstracts reading the SPD ROM
1055 ingredients that are used by the memory initialization mechanism:
1058 static inline int spd_read_byte(unsigned device, unsigned address)
1061 This function, defined in \texttt{auto.c}, directly maps it's calls to
1062 \texttt{smbus\_read\_byte()} calls if SPD ROM information is read via
1066 static inline int spd_read_byte(unsigned device, unsigned address)
1068 return smbus_read_byte(device & 0xff, address);
1072 If there is no SPD ROM available in the system design, this function
1073 keeps an array of SPD ROM information hard coded per logical RAM module.
1074 It returns the ´faked' SPD ROM information using device and address
1075 as indices to this array.
1078 \subsection {IRQ Tables}
1080 Mainboards that provide an IRQ table should have the following two
1081 variables set in their \texttt{Config.lb} file:
1084 default HAVE_PIRQ_TABLE=1
1085 default IRQ_SLOT_COUNT=7
1088 This will make LinuxBIOS look for the file \\
1089 \texttt{freebios2/src/mainboard/<vendor>/<mainboard>/irq\_tables.c} which
1090 contains the source code definition of the IRQ table. LinuxBIOS corrects
1091 small inconsistencies in the IRQ table during startup (checksum and
1092 number of entries), but it is not yet writing IRQ tables in a completely
1095 \textbf{NOTE:} To get Linux to understand and actually use the IRQ
1096 table, it is not always a good idea to specify the vendor and device id
1097 of the actually present interrupt router device. Linux 2.4 for example
1098 does not know about the interrupt router of the AMD8111 southbridge. In
1099 such cases it is advised to choose the vendor/device id of a compatible
1100 device that is supported by the Linux kernel. In case of the AMD8111
1101 interrupt router it is advised to specify the AMD768/Opus interrupt
1102 controller instead (vendor id=\texttt{0x1022}, device id=\texttt{0x7443})
1104 \subsection {MP Tables}
1106 LinuxBIOS contains code to create MP tables conforming the
1107 Multiprocessor Specification V1.4. To include an MP Table in a LinuxBIOS
1108 image, the following configuration variables have to be set (in the
1109 mainboard specific configuration file
1110 \texttt{freebios2/src/mainboard/<vendor><mainboard>/Config.lb}):
1113 default CONFIG_SMP=1
1114 default CONFIG_MAX_CPUS=1 # 2,4,..
1115 default HAVE_MP_TABLE=1
1118 LinuxBIOS will then look for a function for setting up the MP table in
1119 the file \texttt{freebios2/src/mainboard<vendor>/<mainboard>/mptable.c}:
1122 void *smp_write_config_table(void *v, unsigned long * processor_map)
1125 MP Table generation is still somewhat static, i.e. changing the bus
1126 numbering will force
1127 you to adopt the code in mptable.c. This is subject to change in future
1130 \subsection {ACPI Tables}
1132 There is initial ACPI support in LinuxBIOS now. Currently the only gain with
1133 this is the ability to use HPET timers in Linux. To achieve this, there is a
1134 framework that can generate the following tables:
1142 To enable ACPI in your LinuxBIOS build, add the following lines to your
1143 configuration files:
1145 uses HAVE_ACPI_TABLES
1147 option HAVE_ACPI_TABLES=1
1150 To keep Linux doing it's pci ressource allocation based on IRQ tables and MP
1151 tables, you have to specify the kernel parameter \texttt{pci=noacpi} otherwise
1152 your PCI devices won't get interrupts.
1153 It's likely that more ACPI support will follow, when there is need for certain
1157 LinuxBIOS has three different methods of handling POST codes. They can
1158 be triggered using configuration file options.
1161 \emph{Ignore POST completely}. No early code debugging is possible with
1162 this setting. Set the configuration variable \texttt{NO\_POST} to
1163 \texttt{1} to switch off all POST handling in LinuxBIOS.
1165 \emph{Normal IO port 80 POST}. This is the default behavior of
1166 LinuxBIOS. No configuration variables have to be set. To be able to see
1167 port 80 POST output, you need a POST expansion card for ISA or PCI. Port
1168 80 POST allows simple debugging without any other output method
1169 available (serial interface or VGA display)
1172 This option allows to push POST messages to the serial interface instead
1173 of using IO ports. \textbf{NOTE:} The serial interface has to be
1174 initialized before serial POST can work. To use serial POST, set the
1175 configuration variable \texttt{CONFIG\_SERIAL\_POST} to the value 1.
1179 \subsection{HDT Debugging}
1180 If you are debugging your LinuxBIOS code with a Hardware Debug Tool
1181 (HDT), you can find the source code line for a given physical EIP
1182 address as follows: Look the address up in the file linuxbios.map. Then
1183 search the label Lxx in the file auto.inc created by romcc. The original
1184 source code file and line number is mentioned in auto.inc.
1187 \subsection{Device Drivers}
1188 With only a few data structures LinuxBIOS features a simple but flexible
1189 device driver interface. This interface is not intended for autonomously
1190 driving the devices but to initialize all system components so that they
1191 can be used by the booted operating system.
1193 Since nowadays most systems are PCI centric, the data structures used
1194 are tuned towards (onboard and expansion bus) PCI devices. Each driver
1195 consists of at least two structures.
1197 The \texttt{pci\_driver} structure maps PCI vendor/device id pairs to a
1198 second structure that describes a set of functions that together
1199 initialize and operate the device:
1202 static void adaptec_scsi_init(struct device *dev)
1206 static struct device_operations lsi_scsi_ops = {
1207 .read_resources = pci_dev_read_resources,
1208 .set_resources = pci_dev_set_resources,
1209 .enable_resources = pci_dev_enable_resources,
1210 .init = lsi_scsi_init,
1213 static struct pci_driver lsi_scsi_driver __pci_driver = {
1214 .ops = &lsi_scsi_ops,
1220 By separating the two structures above, M:N relations between compatible
1221 devices and drivers can be described. The driver source code containing
1222 above data structures and code have to be added to a LinuxBIOS image
1223 using the driver keyword in the mainboard specific configuration file \\
1224 \texttt{freebios2/src/mainboard/<vendor>/<mainboard>/Config.lb}:
1230 \subsection{Bus Bridges}
1232 Currently all bridges supported in the LinuxBIOS2 tree are transparent
1233 bridges. This means, once the bridge is initialized, it's remote devices
1234 are visible on one of the PCI buses without special probing. LinuxBIOS
1235 supports also bridges that are nontransparent. The driver support code
1236 can provide a \texttt{scan\_bus} function to scan devices behind the bridge.
1238 \subsection{CPU Reset}
1239 When changing speed and width of hypertransport chain connections
1240 LinuxBIOS has to either assert an LDTSTOP or a reset to make the changes
1241 become active. Additionally Linux can do a firmware reset, if LinuxBIOS
1242 provides the needed infrastructure. To use this capability, define the
1243 option \texttt{HAVE\_HARD\_RESET} and add an object file specifying the
1244 reset code in your mainboard specific configuration file
1245 \texttt{freebios2/src/mainboard/$<$vendor$>$/$<$mainboard$>$/Config.lb}:
1248 default HAVE_HARD_RESET=1
1252 The C source file \texttt{reset.c} (resulting in \texttt{reset.o}
1253 during compilation) shall define the following function to take care
1254 of the system reset:
1257 void hard_reset(void);
1260 See \texttt{freebios2/src/mainboard/arima/hdama/reset.c} for an example
1266 % 11. LinuxBIOS Internals
1269 \section{LinuxBIOS Internals}
1270 This chapter covers some of the internal structures and algorithms of
1271 LinuxBIOS that have not been mentioned so far.
1273 \subsection{Code Flow}
1277 \includegraphics[scale=0.7]{codeflow.pdf}
1278 \caption{LinuxBIOS rough Code Flow}
1279 \label{fix:codeflow}
1284 \subsection{Fallback mechanism}
1285 LinuxBIOS provides a mechanism to pack two different LinuxBIOS builds
1286 within one LinuxBIOS ROM image. Using the system CMOS memory LinuxBIOS
1287 determines whether the last boot with a default image succeeded and
1288 boots a failsafe image on failure. This allows insystem testing without
1289 the risk to render the system unusable. See
1290 \texttt{freebios2/src/mainboard/arima/hdama/failover.c} for example
1291 code. The fallback mechanism can be used with the \texttt{cmos\_util}.
1293 \subsection{(Un) Supported Standards}
1294 LinuxBIOS supports the following standards
1296 \item Multiprocessing Specification (MPSPEC) 1.4
1297 \item IRQ Tables (PIRQ)
1298 \item ACPI (initial support on AMD64)
1301 However, the following standards are not supported until now, and will
1302 probably not be supported in future revisions:
1307 \subsection{LinuxBIOS table}
1308 LinuxBIOS stores information about the system in a data structure called
1309 the LinuxBIOS table. This table can be read under Linux using the tool
1310 lxbios from the Lawrence Livermore National Laboratory.
1312 Get more information about lxbios and the utility itself at
1313 \url{http://www.llnl.gov/linux/lxbios/lxbios.html}
1315 \subsection{ROMCC limitations}
1316 ROMCC, part of the LinuxBIOS project, is a C compiler that translates to
1317 completely rommable code. This means the resulting code does not need
1318 any memory to work. This is one of the major improvements in LinuxBIOS
1319 V2, since it allows almost all code to be written in C. DRAM
1320 initialization can be factored and reused more easily among mainboards
1323 Since no memory is available during this early initialization point,
1324 romcc has to map all used variables in registers for the time being.
1325 Same applies for their stack usage. Generally the less registers are
1326 used up by the algorithms, the better code can be factored, resulting in
1327 a smaller object size. Since getting the best register usage is an NP
1328 hard problem, some heuristics are used to get reasonable translation
1329 time. If you run out of registers during compilation, try to refactor
1332 \subsection{CMOS handling}
1333 LinuxBIOS can use the mainboard's CMOS memory to store information
1334 defined in a data structure called the CMOS table . This information
1335 contains serial line speed, fallback boot control, output verbosity,
1336 default boot device, ECC control, and more. It can be easily enhanced by
1337 enhancing the CMOS table. This table, if present, is found at
1338 \texttt{freebios2/src/mainboard/$<$vendor$>$/$<$mainboard$>$/cmos.layout}.
1339 It describes the available options, their possible values and their
1340 position within the CMOS memory. The layout file looks as follows:
1342 # startbit length config configID name
1347 # configid value human readable description
1359 To change CMOS values from a running Linux system, use the
1360 \texttt{cmos\_util}, provided by Linux Networks as part of the LinuxBIOS
1361 utilities suite. Get it at
1362 \textit{ftp://ftp.lnxi.com/pub/linuxbios/utilities/}
1364 \subsection {Booting Payloads}
1365 LinuxBIOS can load a payload binary from a Flash device or IDE. This
1366 payload can be a boot loader, like FILO or Etherboot, a kernel image, or
1367 any other static ELF binary.
1369 To create a Linux kernel image, that is bootable in LinuxBIOS, you have
1370 to use mkelfImage. The command line I used, looks like follows:
1373 objdir/sbin/mkelfImage t bzImagei386 kernel /boot/vmlinuz \
1374 commandline="console=ttyS0,115200 root=/dev/hda3" \
1375 initrd=/boot/initrd output vmlinuz.elf
1379 This will create the file \texttt{vmlinuz.elf} from a distribution
1380 kernel, console redirected to the serial port and using an initial
1383 \subsection{Kernel on dhcp/tftp}
1385 One possible scenario during testing is that you keep your kernel (or
1386 any additional payload) on a different machine on the network. This can
1387 quickly be done using a DHCP and TFTP server.
1389 Use for example following \texttt{/etc/dhcpd.conf} (adapt to your
1393 subnet 192.168.1.0 netmask 255.255.255.0 {
1394 range 192.168.1.0 192.168.1.31;
1395 option broadcastaddress 192.168.1.255;
1398 ddnsupdatestyle adhoc;
1401 hardware ethernet 00:04:76:EA:64:31;
1402 fixedaddress 192.168.1.24;
1403 filename "vmlinuz.elf";
1408 Additionally you have to run a \texttt{tftp} server. You can start one
1409 using \texttt{inetd}. To do this, you have to remove the comment from
1410 the following line in \texttt{/etc/inetd.conf}:
1413 tftp dgram udp wait root /usr/sbin/in.tftpd in.tftpd -s /tftpboot
1416 Then put your kernel image \texttt{vmlinuz.elf} in \texttt{/tftpboot} on
1417 the \texttt{tftp} server.
1423 % 12. Advanced Device Initialization Mechanisms
1426 \section{Advanced Device Initialization Mechanisms}
1428 Like software, today's hardware is getting more and more complex. To
1429 stay flexible many hardware vendors start breaking hardware
1430 compatibility to old standards like VGA. Thus, VGA is still supported by
1431 most cards, but emulation has to be enabled by the firmware for the
1432 device to operate properly. Also, many expansion cards are small
1433 discrete systems that have to initialize attached ram, download
1434 controller firmware and similar. Without this initialization, an
1435 operating system can not take advantage of the hardware, so there needs
1436 to be a way to address this issue. There are several alternatives:
1438 \subsection{Native LinuxBIOS Support}
1440 For some devices (ie Trident Cyberblade 3d) there is native LinuxBIOS
1441 support This means there is a small driver bound to the PCI id of the
1442 device that is called after PCI device ressources are allotted.
1447 \item minimal driver
1453 \item need an additional driver
1454 \item viable for onboard devices only
1455 \item not flexible for pci cards
1458 \subsection{Using Native Linux Support}
1460 A simple way of getting a whole lot of drivers available for LinuxBIOS
1461 is to reuse Linux drivers by putting a Linux kernel to flash. This
1462 works, because no drivers are needed to get the Linux kernel (as opposed
1463 to store the kernel on a harddisk connected to isa/scsi/raid storage)
1467 \item large number of open source drivers
1472 \item need Linux in Flash (BLOAT!)
1473 \item drivers expect devices to be initialized (LSI1020/1030)
1475 \item large flash needed (4MBit minimum, normal operations 8+ MBit)
1479 \subsection{Running X86 Option ROMs}
1481 Especially SCSI/RAID controllers and graphics adapters come with a
1482 special option rom. This option rom usually contains x86 binary code
1483 that uses a legacy PCBIOS interface for device interaction. If this code
1484 gets executed, the device becomes operable in Linux and other operating
1489 \item really flexible
1490 \item no need for additional drivers on firmware layer
1491 \item large number of supported devices
1496 \item non-x86 platforms need complex emulation
1497 \item x86 platforms need legacy API
1498 \item outdated concept
1502 \subsection{Running Open Firmware Option ROMs}
1504 Some PCI devices come with open firmware option roms. These devices are
1505 normally found in computers from SUN, Apple or IBM. Open Firmware is a
1506 instruction set architecture independent firmware standard that allows
1507 device specific initialization using simple, small, but flexible
1508 bytecode that runs with minimal footprint on all architectures that have
1509 an Open Firmware implementation.
1511 There is a free Open Firmware implementation available, called OpenBIOS,
1512 that runs on top of LinuxBIOS. See www.openbios.org
1516 \item architecture independence
1517 \item small footprint
1518 \item clean concept, less bugs
1523 \item only small number of devices come with OpenFirmware capable option roms
1535 LinuxBIOS only cares about lowlevel machine initialization, but also has
1536 very simple mechanisms to boot a file either from FLASHROM or IDE. That
1537 file, possibly a Linux Kernel, a boot loader or Etherboot, are called
1538 payload, since it is the first software executed that does not cope with
1539 pure initialization.
1543 Flash devices are commonly used in all different computers since unlike
1544 ROMs they can be electronically erased and reprogrammed.
1553 \section{Bibliography}
1554 \subsection{Additional Papers on LinuxBIOS}
1558 \textit{\url{http://www.linuxnetworx.com/products/linuxbios_white_paper.pdf}}
1561 \textit{\url{http://www.linuxbios.org/papers/}}
1563 \textit{\url{http://www.lysator.liu.se/upplysning/fa/linuxbios.pdf}}
1565 \textit{\url{http://portal.acm.org/citation.cfm?id=512627}}
1571 \item Etherboot: \textit{\url{http://www.etherboot.org/}}
1572 \item Filo: \textit{\url{http://te.to/~ts1/filo/}}
1573 \item OpenBIOS: \textit{\url{http://www.openbios.org/}}