3. Resolving installation and configuration problems

This section describes some of the most common failure modes for the PCMCIA subsystem. Try to match your symptoms against the examples. This section only describes general failures that are not specific to a particular client driver or type of card.

Before trying to diagnose a problem, you have to know where your system log is kept (see Notes about specific Linux distributions). You should also be familiar with basic diagnostic tools like dmesg and lsmod. Also, be aware that most driver components (including all the kernel modules) have their own individual man pages.

In 3.1.15 and later releases, the debug-tools subdirectory of the PCMCIA source tree has a few scripts to help diagnose some of the most common configuration problems. The test_setup script checks your PCMCIA installation for completeness. The test_network and test_modem scripts will try to diagnose problems with PCMCIA network and modem cards. These scripts can be particularly helpful if you are unfamiliar with Linux and are not sure how to approach a problem.

Try to define your problem as narrowly as possible. If you have several cards, try each card in isolation, and in different combinations. Try cold Linux boots, versus warm boots from Windows. Compare booting with cards inserted, versus inserting cards after boot. If you normally use your laptop docked, try it undocked. And sometimes, two sockets will behave differently.

For debugging problems in the device configuration scripts, it may be useful to start cardmgr with the ``-v'' option. With a 3.1.23 or later PCMCIA package, this will cause most important script actions to be recorded in the system log.

It is nearly impossible to debug driver problems encountered when attempting to install Linux via a PCMCIA device. Even if you can identify the problem based on its symptoms, installation disks are difficult to modify, especially without access to a running Linux system. Customization of installation disks is completely dependent on the choice of Linux distribution, and is beyond the scope of this document. In general, the best course of action is to install Linux using some other means, obtain the latest drivers, and then debug the problem if it persists.

3.1 Base PCMCIA kernel modules do not load

Symptoms:

• Kernel version mismatch errors are reported when the PCMCIA startup script runs.
• After startup, lsmod does not show any PCMCIA modules.
• cardmgr reports ``no pcmcia driver in /proc/devices'' in the system log.

Kernel modules contain version information that is checked against the current kernel when a module is loaded. The type of checking depends on the setting of the CONFIG_MODVERSIONS kernel option. If this is false, then the kernel version number is compiled into each module, and insmod checks this for a match with the running kernel. If CONFIG_MODVERSIONS is true, then each symbol exported by the kernel is given a sort of checksum. These codes are all compared against the corresponding codes compiled into a module. The intent was for this to make modules less version-dependent, because the checksums would only change if a kernel interface changed, and would generally stay the same across minor kernel updates. In practice, the checksums have turned out to be even more restrictive, because many kernel interfaces depend on compile-time kernel option settings. Also, the checksums turned out to be an excessively pessimistic judge of compatibility.

The practical upshot of this is that kernel modules are closely tied to both the kernel version, and the setting of many kernel configuration options. Generally, a set of modules compiled for one 2.2.19 kernel will not load against some other 2.2.19 kernel unless special care is taken to ensure that the two were built with similar configurations. This makes distribution of precompiled kernel modules a tricky business.

You have several options:

• If you obtained precompiled drivers as part of a Linux distribution, verify that you are using an unmodified kernel as supplied with that distribution. If you intend to use precompiled modules, you generally must stick with the corresponding kernel.
• If you have reconfigured or upgraded your kernel, you will probably need to compile and install the PCMCIA package from scratch. This is easily done if you already have the kernel source tree installed. See Compilation and installation for detailed instructions.
• In some cases, incompatibilities in other system components can prevent correct loading of kernel modules. If you have upgraded your own kernel, pay attention to the ``minimal requirements'' for module utilities and binutils listed in the Documentation/Changes file in the kernel source code tree.

3.2 Some client driver modules do not load

Symptoms:

• The base modules (pcmcia_core, ds, i82365) load correctly.
• Inserting a card gives a high beep + low beep pattern.
• cardmgr reports version mismatch errors in the system log.

Some of the driver modules require kernel services that may or may not be present, depending on kernel configuration. For instance, the SCSI card drivers require that the kernel be configured with SCSI support, and the network drivers require a networking kernel. If a kernel lacks a necessary feature, insmod may report undefined symbols and refuse to load a particular module. Note that insmod error messages do not distinguish between version mismatch errors and missing symbol errors.

Specifically:

• The serial client driver serial_cs requires the kernel serial driver to be enabled with CONFIG_SERIAL. This driver may be built as a module.
• Support for multiport serial cards or multifunction cards that include serial or modem devices requires CONFIG_SERIAL_SHARE_IRQ to be enabled.
• The SCSI client drivers require that CONFIG_SCSI be enabled, along with the appropriate top level driver options (CONFIG_BLK_DEV_SD, CONFIG_BLK_DEV_SR, etc for 2.2 and later kernels). These may be built as modules.
• The network client drivers require that CONFIG_INET is enabled. Kernel networking support cannot be compiled as a module.
• The token-ring client requires that the kernel be compiled with CONFIG_TR enabled.

There are two ways to proceed:

• Rebuild your kernel with the necessary features enabled.
• If the features have been compiled as modules, then modify /etc/pcmcia/config to preload these modules.

The /etc/pcmcia/config file can specify that additional modules need to be loaded for a particular client. For example, for the serial driver, one would use:

device "serial_cs"
  class "serial" module "misc/serial", "serial_cs"

Module paths are specified relative to the top-level module directory for the current kernel version; if no relative path is given, then the path defaults to the pcmcia subdirectory.

3.3 ISA interrupt scan failures

Symptoms:

• The system locks up when the PCMCIA drivers are loaded, even with no cards present.
• The system log shows a successful host controller probe just before the lock-up, but does not show interrupt probe results.

After identifying the host controller type, the socket driver probes for free ISA bus interrupts. The probe involves programming the controller for each apparently free interrupt, then generating a ``soft'' interrupt, to see if the interrupt can be detected correctly. In some cases, probing a particular interrupt can interfere with another system device.

The reason for the probe is to identify interrupts which appear to be free (i.e., are not reserved by any other Linux device driver), yet are either not physically wired to the host controller, or are connected to another device that does not have a driver.

In the system log, a successful probe might look like:

Intel PCIC probe:
  TI 1130 CardBus at mem 0x10211000, 2 sockets
  ...
  ISA irqs (scanned) = 5,7,9,10 status change on irq 10

There are two ways to proceed:

• The ISA interrupt probe can be restricted to a list of interrupts using the irq_list parameter for the socket drivers. For example, ``irq_list=5,9,10'' would limit the scan to three interrupts. All 16-bit PCMCIA devices will be restricted to using these interrupts (assuming they pass the probe). You may need to use trial and error to find out which interrupts can be safely probed.
• The interrupt probe can be disabled entirely by loading the socket driver with the ``do_scan=0'' option. In this case, a default interrupt list will be used, which just avoids interrupts already allocated for other devices.

In either case, the probe options can be specified using the PCIC_OPTS definition in the PCMCIA startup script, for example:

PCIC_OPTS="irq_list=5,9,10"

It should be noted that /proc/interrupts is completely useless when it comes to diagnosing interrupt probe problems. The probe is sensible enough to never attempt to use an interrupt that is already in use by another Linux driver. So, the PCMCIA drivers are already using all the information in /proc/interrupts. Depending on system design, an inactive device can still occupy an interrupt and cause trouble if it is probed for PCMCIA.

3.4 IO port scan failures

Symptoms:

• The system locks up when cardmgr is first started. For 3.1.24, the lockup happens even with no cards present; for 3.1.25, a card must be inserted.
• The system log shows a successful host controller probe, including interrupt probe results, but does not show IO probe results.
• In some cases, the IO probe will succeed, but report large numbers of random exclusions.

When cardmgr processes IO port ranges listed in /etc/pcmcia/config.opts, the kernel probes these ranges to detect latent devices that occupy IO space but are not associated with a Linux driver. The probe is read-only, but in rare cases, reading from a device may interfere with an important system function, resulting in a lock-up.

Your system user's guide may include a map of system devices, showing their IO and memory ranges. These can be explicitly excluded in config.opts.

Alternatively, if the probe is unreliable on your system, it can be disabled by setting CORE_OPTS to ``probe_io=0''. In this case, you should be very careful to specify only genuinely available ranges of ports in config.opts, instead of using the default settings.

3.5 Memory probe failures

Symptoms:

• The core drivers load correctly when no cards are present, with no errors in the system log.
• The system freezes and/or reboots as soon as any card is inserted, before any beeps are heard.

Or alternately:

• All card insertions generate a high beep followed by a low beep.
• All cards are identified as ``anonymous memory cards''.
• The system log reports that various memory ranges have been excluded.

The core modules perform a memory scan at the time of first 16-bit card insertion. This scan can potentially interfere with other memory mapped devices. Also, pre-3.0.0 driver packages perform a more aggressive scan than more recent drivers. The memory window is defined in /etc/pcmcia/config.opts. The default window is large, so it may help to restrict the scan to a narrower range. Reasonable ranges to try include 0xd0000-0xdffff, 0xc0000-0xcffff, 0xc8000-0xcffff, or 0xd8000-0xdffff.

If you have DOS or Windows PCMCIA drivers, you may be able to deduce what memory region those drivers use. Note that DOS memory addresses are often specified in ``segment'' form, which leaves off the final hex digit (so an absolute address of 0xd0000 might be given as 0xd000). Be sure to add the extra digit back when making changes to config.opts.

Changing BIOS settings affecting how devices are mapped can sometimes be useful. Try changing settings for BIOS shadowing, or "Plug and Play OS support".

In unusual cases, a memory probe failure can indicate a timing register setup problem with the host controller. See the Startup options section for information about dealing with common timing problems. This really only applies to ISA-to-PCMCIA bus bridges.

• cs: warning: no high memory space available!

CardBus bridges can allocate memory windows outside of the 640KB-1MB ``memory hole'' in the ISA bus architecture. It is generally a good idea to configure CardBus bridges to use high memory windows, because these are unlikely to conflict with other devices. Also, CardBus cards may require large memory windows, which may be difficult or impossible to fit into low memory. Card Services will preferentially allocate windows in high memory for CardBus bridges, if both low and high memory windows are defined in config.opts. The default config.opts includes several candidate high memory windows, one of which will work in most cases.

3.6 Failure to detect card insertions and removals

Symptoms:

• Cards are detected and configured properly if present at boot time.
• The drivers do not respond to insertion and removal events, either by recording events in the system log, or by beeping.

In most cases, the socket driver (i82365 or tcic) will automatically probe and select an appropriate interrupt to signal card status changes. The automatic interrupt probe doesn't work on some Intel-compatible controllers, including Cirrus chips and the chips used in some IBM ThinkPads. If a device is inactive at probe time, its interrupt may also appear to be available. In these cases, the socket driver may pick an interrupt that is used by another device.

With the i82365 and tcic drivers, the irq_list option can be used to limit the interrupts that will be tested. This list limits the set of interrupts that can be used by PCMCIA cards as well as for monitoring card status changes. The cs_irq option can also be used to explicitly set the interrupt to be used for monitoring card status changes.

If you can't find an interrupt number that works, there is also a polled status mode: both i82365 and tcic will accept a poll_interval=100 option, to poll for card status changes once per second. This option should also be used if your system has a shortage of interrupts available for use by PCMCIA cards. Especially for systems with more than one host controller, there is little point in dedicating interrupts for monitoring card status changes.

All these options should be set in the PCIC_OPTS= line in either /etc/rc.d/rc.pcmcia or /etc/sysconfig/pcmcia, depending on your site setup.

3.7 Interrupt delivery problems

Symptoms:

• Cards appear to be configured successfully, but don't work.
• Serial and modem cards may respond very sluggishly.
• Network cards may report ``interrupt(s) dropped'', and/or transmit timeouts.

The most simple interrupt delivery problems are due to conflicts with other system devices. These can generally be resolved by excluding problem interrupts in /etc/pcmcia/config.opts. To test, just exclude interrupts one by one until either the problem is fixed or you run out of interrupts. If no interrupts work, then device conflicts are probably not the problem.

For CardBus bridges, a variety of other interrupt delivery issues may come into play. For a complete discussion, see PCI interrupt delivery problems.

3.8 System resource starvation

Symptoms:

• When a card is inserted, it is identified correctly but cannot be configured (high/low beep pattern).
• One of the following messages will appear in the system log:

  RequestIO: Resource in use
  RequestIRQ: Resource in use
  RequestWindow: Resource in use
  GetNextTuple: No more items
  could not allocate nn IO ports for CardBus socket n
  could not allocate nnK memory for CardBus socket n
  could not allocate interrupt for CardBus socket n
  

Interrupt starvation often indicates a problem with the interrupt probe (see Interrupt scan failures). In some cases, the probe will seem to work, but only report one or two available interrupts. Check your system log to see if the scan results look sensible. Disabling the probe and selecting interrupts manually should help.

If the interrupt probe is not working properly, the socket driver may allocate an interrupt for monitoring card insertions, even when interrupts are too scarce for this to be a good idea. You can switch the controller to polled mode by setting PCIC_OPTS to ``poll_interval=100'. Or, if you have a CardBus controller and an older version of the PCMCIA drivers, try ``pci_csc=1'', which selects a PCI interrupt (if available) for card status changes.

In some cases, kernel misconfiguration can also produce an apparent interrupt shortage. On 2.4 and later kernels, if CONFIG_ISA is not enabled, then the PCMCIA drivers will assume no ISA bus interrupts are available.

IO port starvation is fairly uncommon, but sometimes happens with cards that require large, contiguous, aligned regions of IO port space, or that only recognize a few specific IO port positions. The default IO port ranges in /etc/pcmcia/config.opts are normally sufficient, but may be extended. If this is the problem, try uncommenting the ``include port 0x1000-0x17ff'' line in config.opts. In rare cases, starvation may indicate that the IO port probe failed (see IO port scan failures).

Memory starvation is also uncommon with the default memory window settings in config.opts. CardBus cards may require larger memory regions than typical 16-bit cards. Since CardBus memory windows can be mapped anywhere in the host's PCI address space (rather than just in the 640K-1MB ``hole'' in PC systems), it is helpful to specify large memory windows in high memory, such as 0xa0000000-0xa0ffffff.

3.9 Resource conflict only with two cards inserted

Symptoms:

• Two cards each work fine when used separately.
• When both cards are inserted, only one works.

This usually indicates a resource conflict with a system device that Linux does not know about. PCMCIA devices are dynamically configured, so, for example, interrupts are allocated as needed, rather than specifically assigned to particular cards or sockets. Given a list of resources that appear to be available, cards are assigned resources in the order they are configured. In this case, the card configured last is being assigned a resource that in fact is not free.

Check the system log to see what resources are used by the non-working card. Exclude these in /etc/pcmcia/config.opts, and restart the cardmgr daemon to reload the resource database.

3.10 Device configuration does not complete

Symptoms:

• When a card is inserted, exactly one high beep is heard.
• Subsequent card insertions and removals may be ignored.

This indicates that the card was identified successfully, however, cardmgr has been unable to complete the configuration process for some reason. The most likely reason is that a step in the card setup script has blocked. A good example would be the network script blocking if a network card is inserted with no actual network hookup present.

To pinpoint the problem, you can manually run a setup script to see where it is blocking. The scripts are in the /etc/pcmcia directory. They take two parameters: a device name, and an action. The cardmgr daemon records the configuration commands in the system log. For example, if the system log shows that the command ``./network start eth0'' was the last command executed by cardmgr, the following command would trace the script:

sh -x /etc/pcmcia/network start eth0