EPICS SIS3801 and SIS3820 Drivers

Contents

• Overview
• Hardware configuration
• Software architecture
• Multichannel scaler (MCS) operation
• Scaler record operation
• Startup script

Overview

These drivers support the SIS3801 and SIS3820 multi-channel scalers from Struck Innovative Systems. The SIS3801 was formerly sold as the model STR7201 from a predecessor company called Struck. All are VME modules with 8 or 32 scaler input channels. In this document the term SIS38XX includes all of the above modules, except where explicitly noted. Each input can count at 100MHz or 200MHz depending on model. It has an external channel advance input (called Next) which causes the the current scaler values to be copied to a FIFO, clears the scalers and resumes counting. The scalers are implemented in a dual-bank design, so that the switching time is less than 4 nsec.

The drivers are written to support the use of the SIS38XX as a multichannel scaler. Up to 32 mca records (or waveform records) can be connected to each unit, each record corresponding to one of the signal inputs. There are additional records to control start and stop acquisition of the device, control the dwell time and channel advance source, etc.

The driver also supports using the SIS38XX as a conventional EPICS scaler (like the Joerger scaler) with the scaler record. The SIS3801 does not support counting for a preset time or number of counts, so the preset scaler operation is emulated using the FIFO. The SIS3820 does support counting for a preset counts, and so directly supports the scaler mode of operation without emulation.

The SIS38XX hardware itself is not a multichannel scaler in the conventional sense, since it simply contains a FIFO of scaler counts, rather than an array of memory locations which hold the counts for each dwell cycle. The driver for the SIS38XX simulates a conventional MCS by allocating a memory buffer which contains the counts in each dwell period. The driver copies the counts from the FIFO to this memory buffer. The SIS3801 has a limited amount of FIFO memory (128KB to 512KB), while the SIS3820 comes standard with 64MB, and optionally has 512MB. The SIS3820 also maps a large block (8MB) of the FIFO memory to the VME bus, so it can effectively use DMA operations. The SIS3801 only maps 256 bytes to the VME bus, so DMA cannot be effectively used.

Each FIFO word contains the counts for one input signal for one dwell period. On the SIS3801 the FIFO can thus buffer anywhere from 1024 dwell cycles (32 active inputs, 128KB FIFO) to 128K dwell cycles (1 active input, 512KB FIFO). The minimum dwell time is in the range .5-4 microseconds, depending upon the number of active inputs. VME interrupts are implemented, so that FIFO readout is done automatically when the FIFO is half full. Even with interrupts there is a possibility of FIFO overflow with very short dwell times and small FIFO sizes. Users whose applications require short dwell times and a large number of active inputs should purchase large FIFO chips to prevent this. The FIFO chip can be replaced in the field. The driver determines the FIFO size automatically. On the SIS3820 the FIFO size is large (64MB to 512MB), and can be read out quickly (~ 40 MB/s) using DMA. Still, at 1 microsecond dwell time and 32 active inputs the data rate is 128MB/s, which is 3 times faster than the FIFO readout speed. Thus, users who need very large time sequences at minimum dwell times should chose the 512MB FIFO option.

Hardware configuration

Hardware Configuration of older STR7201 cards

Jumper J600 on the board must be set to select LCA file 2 (STR7201 Design 1) or LCA file 3 (STR7201 Design 2). With Design 1 selected the scalers are 32 bits wide. With Design 2 selected the scalers are 24 bits wide, and the upper 8 bits of each data word contain information on the channel number, bank, and 2 user bits. The channel number and bank information are redundant since the driver sorts the FIFO output by channel number when copying the data to the buffer and then to the MCA record. However, some applications may want to use the 2 user bits which are latched with each Next cycle, and Design 2 may be selected for those applications.

The driver requires that the board be configured for VME A32 addressing, so jumper EN_A32 must be in place. Address selection is done with jumper J_A11, and switches SW_A32U, SW_A32L, SW_A24 and SW_A16. Note that the factory-default address of 0x33508000 cannot be used with the APS version of the vxWorks board support package since this part of the VME A32 address space is not accessible. Each board takes up 2K bytes of A32 address space.

Hardware Configuration of SIS3801 and SIS3820 cards

The A32 address space must be used. The VME A32 base address must be set to a value that is supported by the board support package being used. The SIS3801 occupies 2KB of address space, while the SIS3820 occupies 16MB of address space.

The SIS3820 supports loading new firmware into the PROM using either of 2 methods:

1. Using an external JTAG programmer, typically a USB device with a Windows laptop. This method requires that pins 1 and 2 be connected on jumper JP570 (factory configuration).
2. Over the VME bus, using a function provided with this EPICS software called sis3820_jtag_prom_epics(). This method requires that pins 2 and 3 be connected on jumper JP570 (must be changed from factory configuration).

The sis3820_jtag_prom_epics() function is called from the vxWorks or iocsh shell as follows:

sis3820_jtag_prom_epics(baseAddress, command, mcsFile)

where:

• baseAddress is the VME address of the SIS3820 card. This is the same address used in the drvSIS3820Config command.
• command is one of the followiing:
  • "R" (Read configuration information)
  • "V" (Verify the current contents of the PROM to see if it is the same as that in mcsFile
  • "P" (Program the contents of mcsFile into the PROM, and then verify the result)
• mcsFile is the relative path to the new firmware file, e.g. "./prom_v0111.mcs"

The file for the latest firmware can be found either on the SIS S8320 firmware Web site or in the mca/mcaApp/SISSrc directory in the mca module distribution.

Software architecture

The driver consists of a base class, drvSIS38XX.cpp, and two derived classes, drvSIS3801.cpp and drvSIS3820.cpp. The base class inherits from asynPortDriver, which is part of asyn. The driver uses the standard device-independent asyn device support for the MCA record. The INP link is of the form INP=@asyn(PORT, ADDR), where PORT is the name of the asyn port that has been created for a specific SIS38XX card, and ADDR is the input signal number (0-31).

Alternatively, one can use standard EPICS waveform records for the array data, rather than the mca record. This uses the standard device support for the waveform record that comes with asyn. The example startup scripts (st_SIS3801.cmd and st_SIS3820.cmd) in iocBoot/iocVxWorks contain examples for loading either mca or waveform records.

There is an EPICS SNL program, SIS38XX_SNL.st, that must be run for each SIS38XX card. This is a very simple program that forces each MCA or waveform record to process when acquisition completes before setting the Acquiring busy record back to 0. It also forces each MCA or waveform record to process when the ReadAll record processes. This logic is much easier to accomplish in an SNL program than in a database, particularly because the number of MCA or waveform records in the database is not fixed, but can be changed in the startup script.

The SIS3801 and SIS3820 support was rewritten for R7-0 of the mca module. In previous releases the driver was "synchronous", meaning that the MCA records blocked while the arrays were read from the FIFO on the VME modules. FIFO reads were done with VME program I/O, which can only do about 1000 32-bit words per ms. This was acceptable if the arrays were small (e.g. 2048 channels) because it only took a few ms to read each array. However, some users were beginning to use very large arrays (e.g. 2,000,000 channels). In this case it took several seconds to read each array. Because the MCA records were synchronous, this blocked execution of other EPICS records and tasks while reading the hardware. This was unacceptable, leading to channel access disconnects and other bad effects.

One way to fix this would be to make the drivers asynchronous, meaning that device support will do the I/O in a separate thread, and the records will not block. This is how other slow drivers for the MCA records work, for example the Canberra AIM and the XIA DXP modules. However, this approach cannot be used with the SIS drivers because they must support the EPICS "scaler" record, and the scaler record does not work properly with asynchronous device suppport.

Instead, the drivers were rewritten so that the FIFO is read out into a driver buffer in a low-priority background thread. When the MCA or waveform records process they now just read the data from this driver buffer using a memcpy() call, which is very fast. Furthermore on the 3820 the driver now reads the FIFO using DMA over the VME bus. This yields a factor of 10 improvement in speed, over 40MB/s, compared to 4MB/s with program I/O.

The new drivers have excellent performance with very large arrays, and arrays with more than 10 million elements can now be used if the VME CPU has sufficient memory. For example, a test was done with the following configuration:

• SIS3820 module with 512MB of FIFO memory
• MVME 5100 CPU with 512MB of memory
• vxWorks 5.4.2 with Andrew Johnson's BSP with DMA support
• 2 active inputs
• 10,000,000 channels per waveform
• 2 waveform records
• Internal channel advance
• 1 microsecond dwell time
• ReadAll.SCAN = Passive

The theoretical time for this acquisition to complete is 10 seconds. The following is the output from the EPICS "camonitor" program on a Linux client looking at the Acquiring PV, and the first 2 channels of each waveform record:

    >camonitor -tc -#2 SIS:3820:mca1 SIS:3820:mca2 SIS:3820:Acquiring 
    SIS:3820:Acquiring             (2011-05-01 11:19:35.050808) Acquiring 
    SIS:3820:mca1                  (2011-05-01 11:19:46.017134) 2 50 50 
    SIS:3820:mca2                  (2011-05-01 11:19:46.017223) 2 0 0 
    SIS:3820:Acquiring             (2011-05-01 11:19:46.017264) Done 
  

So the time between when acquisition started and when the client received the data and the Acquiring=Done status was 10.97 seconds. There is thus less than 1 second of overhead in collecting 2 waveforms of 10,000,000 elements each, which is 80MB of data.

Note that the DMA function calls used in this driver are to the functions provided in Andrew Johnson's library for vxWorks.  It would be very desireable if these functions were included in the EPICS devLib library in EPICS base, so they would work on any EPICS system that can perform DMA. Until this is done users with other DMA libraries will have to write wrapper functions to emulate the API in Andrew Johnson's library.

Multi-channel scaler (MCS) operation

One instance of the database SIS38XX.template is loaded for each SIS38XX card. This database contains the following records:

Acqusition will be stopped for all records connected to a driver whenever one or more of the following conditions is satisfied:

• The StopAll record is processed.
• The PresetReal time is reached.
• The number of channel advances reaches NUseAll.

Input modes

The SIS3801 supports the following input modes:

• Mode 0
  • Input 1: External next pulse
  • Input 2: External user bit 1
  • Input 3: External user bit 2
  • Input 4: Reset
• Mode 1
  • Input 1: External next pulse
  • Input 2: External user bit 1
  • Input 3: Disable counting
  • Input 4: Reset
• Mode 2
  • Input 1: External next pulse
  • Input 2: External user bit 1
  • Input 3: External user bit 2
  • Input 4: Disable counting
• Mode 3
  • Input 4: External test

The SIS3820 supports the following input modes:

• Mode 0
  • Input 1: No function
  • Input 2: No function
  • Input 3: No function
  • Input 4: No function
• Mode 1
  • Input 1: External next pulse
  • Input 2: External user bit 1
  • Input 3: External user bit 2
  • Input 4: Inhibit next pulse
• Mode 2
  • Input 1: External next pulse
  • Input 2: External user bit 1
  • Input 3: Inhibit counting
  • Input 4: Inhibit next pulse
• Mode 3
  • Input 1: External next pulse
  • Input 2: External user bit 1
  • Input 3: External user bit 2
  • Input 4: Inhibit counting
• Mode 4
  • Input 1: Inhibit counting channels 1-8
  • Input 2: Inhibit counting channels 9-16
  • Input 3: Inhibit counting channels 17-24
  • Input 4: Inhibit counting channels 25-32
• Mode 5
  • Input 1: External next pulse
  • Input 2: External HISCAL_START
  • Input 3: No function
  • Input 4: No function
• Mode 6
  • Input 1: External next pulse
  • Input 2: Inhibit counting
  • Input 3: Clear all counters
  • Input 4: External user bit 1
• Mode 7
  • Input 1: Encoder input A
  • Input 2: Encoder input B
  • Input 3: Encoder input I
  • Input 4: Inhibit counting

Output modes

The SIS3801 only supports a single output mode:

• Mode 0
  • Output 5: Copy in progress (CIP)
  • Output 6: FIFO empty
  • Output 5: FIFO half full
  • Output 8: FIFO full

The SIS3820 supports the following output modes:

• Mode 0
  • Output 5: Scaler mode:LNE pulse; MCS mode: Copy in progress (CIP)
  • Output 6: SDRAM empty
  • Output 7: SDRAM threshold
  • Output 8: User output (User LED)
• Mode 1
  • Output 5: Scaler mode:LNE pulse; MCS mode: Copy in progress (CIP)
  • Output 6: Enabled
  • Output 7: 50 MHz
  • Output 8: User output (User LED)
• Mode 2
  • Output 5: Scaler mode:LNE pulse; MCS mode: Copy in progress (CIP)
  • Output 6: 10MHz (20ns pulse)
  • Output 7: 10MHz (20ns pulse)
  • Output 8: User output (User LED)
• Mode 3 (firmware 0x10a and above)
  • Output 5: Scaler mode:LNE pulse; MCS mode: Copy in progress (CIP)
  • Output 6: 10MHz (20ns pulse)
  • Output 7: Selected MUX OUT channel
  • Output 8: User output (User LED)

On the SIS3820 with firmware version 0x111 and above the properties of the LNE/CIP output pulse (Output 5 in all output modes) can be programmed using the records in the following table:

The following shows the MEDM screen for the MCS operation mode of the SIS3820

SIS38XX.adl with SIS3820

Scaler record operation

In addition to support for the MCA record, the driver supports the EPICS scaler record. Switching between scaler mode and MCA mode is handled automatically. It is important, however, that when acquiring in one mode that acquisition be allowed to complete before starting acquisition in the other mode. It is also important when acquiring in MCA mode to disable AutoCount in scaler mode, or else the MCA acquisition may be interrupted. This should be handled automatically by the logic in the database.

The SIS3820 supports true preset-scaler operation, generating a VME interrupt when counting is complete. The SIS3801 does not support this, it simply counts into the FIFO. In scaler mode the driver sets the SIS3801 for an internal channel advance time of 0.01 seconds. It then reads the FIFO 100 times per second and stops as soon as it detects that the preset counts have been reached. This means that it can count for up to 0.01 second longer than the preset value. However, all inputs count for exactly the same time, and the actual time is accurate, so data normalization should be straightforward.

The following shows the MEDM screen for the scaler record with 8 active inputs

scaler_full.adl with SIS3820

Startup script

The mca/iocBoot/iocVxWorks contains example vxWorks startup scripts for the SIS3801 and SIS3820. st_SIS3820.cmd simply invokes st_SIS3820.iocsh, which contains iocsh commands rather than vxWorks shell commands to allow for the use of environment variables which simplifies the code and makes it easier to understand and modify.

There are 2 functions which are intended to be called from the startup script before iocInit, drvSIS3801Config and drvSIS3820Config

drvSIS3801Config(portName,            # The name of the asyn port to be created
                 baseAddress,         # The base VME A32 address
                 interruptVector,     # The VME interrupt vector
                 interruptLevel,      # The VME interrupt level
                 maxChannels,         # The maximum number of channels (time bins) to use
                 maxSignals)          # The number of inputs to use (1-32)


drvSIS3820Config(portName,            # The name of the asyn port to be created
                 baseAddress,         # The base VME A32 address
                 interruptVector,     # The VME interrupt vector
                 interruptLevel,      # The VME interrupt level
                 maxChannels,         # The maximum number of channels (time bins) to use
                 maxSignals,          # The number of inputs to use (1-32)
                 useDMA,              # Enable DMA (1) or disable DMA (0) 
                 fifoBufferWords)     # The number of 32-bit words to read from FIFO into buffer
                                      # Maximum = 2MW = 0x200000

maxSignals is the number of input signals which will be used, in the range 1 to 32. maxSignals controls how many scaler values will be copied to the FIFO each time a Next signal is received. Set this value to the actual number of signals actually being used to conserve FIFO memory on the card and vxWorks buffer memory in the driver. The number of MCA records which can be attached to this card is also controlled by maxSignals.

maxChannels is the maximum number of channels (time points) that will be used in any MCA or waveform record connected to this card. The number of bytes of IOC buffer memory allocated by the driver for this card is maxChans * maxSignals * 4, so set this value to the actual maximum number of channels to be used in any record to conserve memory.