Paranoia Project Introduction

The Paranoia IV library simplifies portable packet command based device programming across platforms and across device interfaces on the same platform; the idea is to make ATAPI/SCSI programming uniform regardless of the platform (Linux, Solaris, NetBSD, FreeBSD, etc) and regardless of the kernel interface on a specific platform (Linux SG, Linux PG, *BSD SCSI ioctl(), etc). The abstraction layers in Paranoia IV are also arranged to allow adding other packet command interfaces (USB will probably be the first) with minimal pain.

Secondly, Paranoia then builds specialized interfaces (eg, CDROM CDDA) on top of these packet interfaces.

Other other libraries also provide this infrastructure; paranoia has several additional goals that necessitated a somewhat different abstraction structure from, eg, libscg:

1. automatic hardware, kernel and device identification and handling; no compile-time system settings.
2. built in extensive and intelligent progress and error reporting
3. complete error correction capabilites offered through high-level functionality-based interfaces

(Note to Paranoia developers; libscg is a great place to look at existing, stable SCSI code for other platforms, albeit with a slightly different design bent. Note to application developers: if you're looking for a simpler, single-layer SCSI transport library, libscg is a good choice)

Paranoia functionality summary

1. provide a portable, packet command interface to SCSI and ATAPI devices that is uniform across platforms and devices. Programming to a Parallel port ATAPI CDROM under Linux should be no different than coding for a SCSI drive under Solaris or Rhapsody. This point is the same as libscg.
2. provide specialized interfaces to device types that both simplify accessing device functionality and add robust error handling and correction to inherently unreliable function (eg, digital audio extraction from audio CDROM drives).
3. provide a uniform method across platforms of identifying and addressing device types and specific named devices.
4. provide detailed and intelligent reporting of system capability, software status and error situations.

Paranoia layer overview

Transport

Basic use

The transport layer looks a like a simpler version of libscg and other transport packages. One allocates a transport device using an open function and an fd already opened and attached* to a given packet command device.

*Some systems, like Linux, provide /dev/sg entries that map one-to-one with actual hardware on the SCSI bus[ses]. When an application opens, for example, /dev/sga, that fs already directly corresponds to the first target on the first SCSI chain. Other systems, such as FreeBSD and Solaris, provide generic SCSI devices that may refer to *any* actual piece of SCSI hardware in the system. The device is first opened, then associated with a specific hardware item.

Allocation returns a transport_device structure that contains state information, as well as function pointers to provide the rest of the transport library's functions. Currently, the pointers returned are for the functions 'reset', 'command', 'special' and 'close'. "command" is used to send raw packet commands to the device in question (packed into a packet_command struct) and returns the result of the command. "reset" is used to try to reset the device to a known state if command processing goes awry. "close" is used to close and destroy the transport_device when it is no longer needed. "special" is an abstraction hack that will be explained in the section on implementation of the command layer. Detailed programming instructions and requirements for the transport layer are described later in this document (under programming).

Using callbacks

At this level (transport), use of Paranoia is very simple. Open, submit commands, get results, then close. The simplest form of error reporting is to not use the callback mechanism built into each command. Pass NULL as the callback argument, and any fatal errors simply return an error result and set errno to one of the errors listed in paranoia_errors.h. Function calls will always run uninterrupted to completion or return an error.

Callbacks provide a way to get additional detailed error information about what caused a failure or get detailed progress information from a command before it finishes. All the functions in each layer of Paranoia provide a callbacks argument to the function call.

A callback is a function passed as an argument a Paranoia call that will be called immediately at the point of an error, non-fatal warning or progress information point. The callback is called with the error/warning/progress number, the current working state at the point the error occurred, and an argument that identifies the type of the passed state. The callback mechanism is described in detail in the programming section later. Application developers may write their own callback function from scratch, or make use of some or all of Paranoia's built-in callback mechanism, described next.

Built in callbacks [Report layer]

Developer's applications can construct their own callbacks. To minimize work (and provide an example of how to use callbacks), Paranoia provides a built in set of callback building blocks. The built in callbacks consist of a message database and state lookup mechanism; the state passed to the callback is matched to the message database, and a detailed message returned in text form. The message database is not hardwired, and may be modified or supplemented by applications.

The callback building blocks extend through all of the layers of Paranoia, not just the transport layer. The building blocks are mutable and need not be used at all; the application can construct its own callbacks (Paranoia can be compiled without any of the callback code), use callbacks only occasionally, or ignore the callback mechanism entirely. Use of the callback mechanisms is covered later.

The device scanning/location layer [Auto layer]

The role of the auto layer is to automate the process of returning a useful transport_device struct matching a desired piece of physical hardware into a single step. Search criteria could be any or several of device name (the trivial case), device type, interface type, vendor, or location in a device sequence.

The auto layer can be fairly complex on platforms with multiple possible interfaces for a given device family. In this case, the auto layer is not only responsible for locating hardware, but also being aware of multiple interface families and matching a request to the right device and low-level transport. Further tasks that fall upon the auto layer include mapping specialized devices to generic devices (eg, mapping /dev/sr0 to /dev/sgb or the like) and associating a specialized device to a specific target on system that allow a generic packet command device to address any physical hardware device on that interface.

In summary, the auto layer is given a search criteria and does all the work necessary to hand an open and ready filedescriptor to the transport layer for transport_device allocation. Auto then returns that transport_device to the application (note that the auto layer wraps the transport_device inside an auto_device which also contains additional information about the device, such as the path of the device entry).

The COMMAND layer

The command layer takes a transport_device and returns a high-level command interface for that device. An example is taking a transport_device representing the raw packet interface to a CDROM and requesting a high-level interface with function calls to control playback or extraction of audio disks.

Programming with Paranoia

This section discusses details and examples of programming with Paranoia using the various layer interfaces. It is not an exhaustive reference but does intend to cover all details of programming. Paranoia developers are encouraged to read and internalize this section. It's just as important as the implementation section to follow; after all, Paranoia is a goal-driven exercise, and the goals are a) the simple interface and b) the most reliable, consistent possible behavior.

Programming to the TRANSPORT layer

"Open()ing" a transport_device

Interaction with the transport layer begins with allocating a transport_device using one of the transport layer's 'open' calls. Each interface family on a machine has its own open function*. For example, to allocate a transport_device for an fd belonging to a Linux generic scsi device, one would call linuxsg_open(fd, callback). Note that only the open() naming is different among interfaces and platforms; all open calls are identical except for the exact name of the function called. The various interfaces and open calls available for given platforms are listed in Appendix A.

*The transport layer contains a few inelegancies that expose minor platform differences; the naming of the xxxxx_open() functions is one of these. This naming approach was taken for two reasons: 1) it keeps the transport layer simple; transport is not designed to make determinations about appropriate device interface. That is the job of the auto layer or user application. 2) Depending on circumstance, more than one device interface might be usable for a given fd on some platforms. In this case being overly clever about hiding the actual interface type behind a naming scheme would be more trouble to the implementor and the developer than elegance is worth.

The xxxxxx_open prototypes (defined in trans_paranoia.h) follow:

transport-device *xxxxxxx_open(int fd, void (*callback)(int,int,void))

For the moment, use of the callback mechanism isn't explained in detail; the construction and use of callbacks and the built in Paranoia callback building blocks is described in "Programming Callbacks". When not using callbacks, pass 'NULL' as the callback argument.

Call xxxxxx_open() with a valid, attached fd; that is, on systems that may associate a generic interface entry with any actual hardware target (eg, on FreeBSD, a generic SCSI device can be associated with any SCSI target). On systems like Linux and NetBSD, this is not an issue as a generic device is associated directly with a specific target.

A failed call to xxxxxx_open() returns NULL and sets errno appropriately. Note that Paranoia defines its own error numbers in order that numbers for a failure are consistent across platforms. This document does not yet include an exhaustive list of possible errors for a command as they may vary from platform to platform. However, Appendix B lists all currently possible Paranoia IV error, warning and progress numbers. Generally, a failure to open [allocate] a filedescriptor into a transport device is the result of mismatching the requested interface with an fd from a device on a different interface (for example, using linuxsg_open() on an fd from a linux PG device).

A successful call to xxxxxx_open() returns a pointer to a transport_device struct that's ready for use. After a successful return, the fd is considered to belong to the transport_device. It will be closed when the transport device is closed.

The transport_device structure

typedef struct transport_device{
  int packettype; /* packet_set: scsi-like, usb, special, etc */
  int iffamily;   /* device interface family used by device */
  int hardtype;   /* worm, cdrom, etc */

  int fd;

  int max_request_bytes;
  int max_response_bytes;
  
  unsigned char *inqbytes;
  char *devmodel;

  int (*close)(struct transport_device *,void (*callback)(int,int,void *));
  int (*reset)(struct transport_device *,void (*callback)(int,int,void *));
  int (*command)(struct transport_device *,packet_command *,
                 void (*callback)(int,int,void *));
  int (*special)(struct transport_device *,int commandset,
                 void (*callback)(int,int,void *));

  void *platform_dev;
  report_base *reportdb;
} transport_device;

Most of this structure is static device information.

packettype element

PACKET_WILDCARD

Not used in a transport_device, but may be used by higher level layers for searching whent he application wishes to specify that any packet type is acceptible.

PACKET_UNKNOWN

Never used in an opened transport_device, but a partially opened transport device passed to a callback may not have the packettype determined yet; in this case the type would be set to PACKET_UNKNOWN.

PACKET_SCSI

The device in question is a real SCSI device and uses SCSI packet commands

PACKET_ATAPI

The device is ATAPI and uses SCSI packet commands. The differences between ATAPI and SCSI packets are very minor (ATAPI is a subset of SCSI)

PACKET_COOKED

The device does not use packet commands at all; it uses a specialized command set accessed through 'special' (introduced later)*)

*The second compromise in the Paranoia abstractions has to do with how to handle devices accessible only through high-level, specialized kernel interfaces, such as the CDROMREADAUDIO ioctl() in Linux and Solaris. The problem is how to minimize non-portable code, but allow applications to access the specialized devices similarly to normal generic devices. In order to avoid platform specific code from spreading into the COMMAND layer, specialized interfaces are implemented and placed into TRANSPORT. Use of specialized devices is described under 'Specialized transport devices'.

iffamily element

iffamily idenitifies what [kernel] device family supports the transport_device. Ideally, all generic SCSI-like devices would fall into the 'IFFAMILY_SCSI_GENERIC', but several OSes use different interfaces for SCSI-like devices depending on how the hardware is attached to the system. Under Linux, for example, ATAPI devices attached to an IDE controller and SCSI devices are supported under a single generic interface (generic SCSI, tagged IFFAMILY_SCSI_GENERIC), put ATAPI drives attached to a paralell port use a different, incompatable interface (IFFAMLY_PARIDE_GENERIC).

This field is provided for informational purposes; it is not necessary to take any notice of it in order to use Paranoia, and the exact family reported for a piece of hardware will necessarily vary from platform to plaform as it reflects the interface abstractions of a given kernel. The interface family setting is sufficient on any given platform to map a transport_device to the actual kernel interface being used for communication, although the precise mappings vary from platform to platform.

paranoia_types.h also defines a list of NULL terminated string names in the array iff_names when the source defines IFFNAMES. These names are addressed using the IFFAMILY_xxxx macro values. Eg, iff_names[IFFAMILY_IDESPECIAL] references the string "specialized IDE". See paranoia_types.h for details.

The currently defined interface families are:

IFFAMILY_WILDCARD

Used when specifying search criteria; not used with the transport layer.

IFFAMILY_UNKNOWN

An open transport_device will never set the iffamily to unknown, but a partially open device passed to a callback may not yet have set its interface family in which case it is set to IFFAMILY_UNKNOWN,

IFFAMILY_OTHER

The interface family was identifiable, but not as a type supported by Paranoia. A usable transport device would not set iffamily to IFFAMILY_OTHER.

IFFAMILY_PROPSPECIAL

The transport_device in question is not a packet command based interface, but a specialized interface of some odd type (eg: The proprietary soundblaster CDROM /dev entry on some systems).

IFFAMILY_SCSISPECIAL

The transport_device is a SCSI device that does not support a packet command interface. This setting is used rather than IFFAMILY_PROPSPECIAL mainly to offer the additional information that the device is SCSI. The auto layer uses this setting internally on some platforms to signify that the device could possibly be mapped to an actual generic device.

IFFAMILY_SCSIGENERIC

The device is a packet-command capable SCSI device (or a device supported by the SCSI interface layer, such as ATAPI under NetBSD). The device may be a dedicated packet command device (such as under Linux), or a specialized device that also allows packet command access (as is usually the case under most BSDs).

IFFAMILY_PARIDESPECIAL

The device is a specialized parallel-port IDE/ATAPI device that does not have a packet command interface. This setting is used rather than IFFAMILY_PROPSPECIAL mainly to offer the additional information that the device is PARIDE. The auto layer uses this setting internally on some platforms to signify that the device could possibly be mapped to an actual generic device.

IFFAMILY_PARIDEGENERIC

The device is a packet-command capable PARIDE device (or a device supported by the PARIDE interface layer). This setting is used rather than IFFAMILY_SCSIGENERIC when the PARIDE device packet command interface on a platform is not compatible with the generic SCSI device interface.

IFFAMILY_PARSCSISPECIAL

By this point you're likely getting the pattern... the device is a parallel-port SCSI device that does not implement a packet command interface. This setting is used rather than IFFAMILY_PROPSPECIAL mainly to offer the additional information that the device is PARSCSI. The auto layer uses this setting internally on some platforms to signify that the device could possibly be mapped to an actual generic device.

IFFAMILY_PARSCSIGENERIC

The device is a packet-command capable PARSCSI device (or a device supported by the PARSCSI interface layer). This setting is used rather than IFFAMILY_SCSIGENERIC when the PARIDE device packet command interface on a platform is not compatible with the generic SCSI device interface.

hardtype element

hardtype identifies what the physical hardware is, eg, CDROM, scanner, hard drive, etc. The element is for informational purposes; it's not necessary to take any notice of it to use Paranoia.

paranoia_types.h also defines a list of NULL terminated string names in the array hard_names when the source defines HARDNAMES. These names are addressed using the HARDTYPE_xxxx macro values. Eg, hard_names[HARDTYPE_CDROM] references the string "cdrom". See paranoia_types.h for details.

The currently defined hardware types are:

HARDTYPE_WILDCARD

Used when specifying search criteria; not used with the transport layer.

HARDTYPE_UNKNOWN

Signifies that Paranoia has not yet been able to determine device type, or was not able to determine device type. At least one ATAPI DVD-ROM has been observed to reject INQUIRY commands; this, for example, would result in the device type being set to HARDTYPE_UNKNOWN.

HARDTYPE_OTHER

The device type was obtainable, but it's not a type recognized by Paranoia.

HARDTYPE_DISK

The device is a fixed disk.

HARDTYPE_CDROM

The device is a CD-ROM, CD-R, CD-RW or DVD-ROM.

HARDTYPE_TAPE

The device is a tape drive.

HARDTYPE_FLOPPY

The device is a floppy drive.

HARDTYPE_SCANNER

The device is a scanner.

fd element

The file descriptor of the generic device. Don't mess with it.

max_request_bytes

The maximum number of data bytes that can be sent in a packet command. The value given here refers only to data bytes and is unaffected by the command size.

max_response_bytes

The maximum number of data bytes that can be returned by a complete packet command. Issuing a command that expects to retrieve more bytes than indicated by this value is most likely to result in an error.

inq and inqbytes

inq is an array of octets containing the result of the device inquiry. The format of the inquiry depends on the packettype and possibly iffamily; for SCSI and SCSI-like devices, this is the data returned by a standard INQUIRY command. inqbytes indicates the number of inquiry bytes available.

devmodel

A null terminated string consisting of the device vendor, model and revision fields. Redundant spaces are removed.

platform_dev

Pointer into an interface specific struct. This is where a specific interface family driver buries information specific to only that interface.

reportdb

Hook for paranoia's built in callback reporting. The message database pointer is put here.

Functions returned in the transport_device

int (*close)(transport_device *,void (*callback)(int,int,void *));

Close, cleanup and free a transport device. This also closes the fd associated with a transport_device. trans_paranoia.h provides a convenience marco so that it's not necessary to dereference the transport_device structure; the macro takes the form:

int trans_close(transport_device *d, void (*callback)(int,int,void*));

close() returns zero on success and nonzero on error, along with setting errno to a value listed in paranoia_errors.h.

int (*reset)(transport_device *,void (*callback)(int,int,void *));

Attempt resetting the transport_device to a known, clean state. Bugs in a partictular platform's middleware (or the firmware of the device itself) can lead to a packet command interface getting well and thoroughly hung. reset() is a last-chance attempt at getting things unstuck. Like close(), trans_paranoia.h implements a convenience macro:

int trans_reset(transport_device *d, void (*callback)(int,int,void*));

reset() returns zero if a reset request could be made, nonzero otherwise along with setting errno to the appropriate value listed in paranoia_errors.h. Note that an interface under which reset is not possible, reset will return nonzero with errno set to P_ERROR_NOSYS.

int (*command)(transport_device *,packet_command *,void (*callback)(int,int,void *));

The real meat of the interface; this is the function call that submits packet commands to a target and returns the results to the application. command() arguments consist of the transport_device, a packet command and a callback function (specify NULLif a callback is not to be used).

Appendix A

• LINUX:
  • generic scsi (sg, including ide-scsi hostadapter emulation): linuxsg_open()
  • generic paride (pg) linuxpg_open()
  • atapi ioctl() cooked (hd) linuxioctl_open()

Appendix B