FLIR Spinnaker areaDetector driver

March 26, 2018

Mark Rivers

University of Chicago

Table of Contents

Introduction

This is an EPICS areaDetector driver for cameras from FLIR (formerly Point Grey). It uses their new Spinnaker SDK, while the ADPointGrey driver uses their older FlyCap2 SDK. It should work with most GigE, 10GigE, and USB 3.0 cameras. It has been tested on GigE (BlackFlyS), and USB 3.0 (Grasshopper3) cameras.

The driver should work on both Windows and Linux. However, so far it has only been tested on Windows. The problem is that the Spinnaker SDK is only supported on Ubuntu 16. There are 2 contraints that prevent it from running on older Linux versions, and on other distributions. The libSpinnaker.so file from their Spinnaker SDK was built with gcc 5.4 and requires GLIBC 2.14 and GLIBCXX 3.4.21. This means it cannot be used with gcc 4.8 on RHEL 7, for example. Furthermore the Spinnaker libraries depend on these shareable libraries: libswscale-ffmpeg.so.3, libavcodec-ffmpeg.so.56, libavutil-ffmpeg.so.54, and libavformat-ffmpeg.so.56. These libraries come from an ffmpeg package that appears to only be available for Ubuntu, and not for Red Hat/Centos distributions. Building ffmpeg from source does not even work because the .so library names and the symbols inside those libraries do not match the ones that the Spinnaker SDK was built with. This is unfortunate, and I have asked FLIR to consider distributing a version of the SDK that will build on RHEL 7.

This driver inherits from ADDriver. It implements many of the parameters in asynNDArrayDriver.h and in ADArrayDriver.h. It also implements a number of parameters that are specific to the Spinnaker cameras. The spinnaker class documentation describes this class in detail.

Camera firmware

FLIR frequently updates the available firmware for each camera to add features and fix bugs. However, they are not very good about putting the link to the latest firmware on their Web site. At present one must e-mail the support team (mv-support@flir.com) and ask them if newer firmware is available for a particular camera, and if so to provide access to it. They normally do this by sending a link to a Dropbox folder.

Properties

The Spinnaker SDK supports cameras that use the GenICam interface. This includes GigE Vision and USB3 Vision cameras. GenICam cameras have an XML file in the camera that describes the complete list of properties that the camera supports. These properties can be accessed in a generic manner. This means that libraries that support GenICam are quite simple to use, since all properties are accessed in a generic way. It supports introspection, so that software can determine which properties are supported, and what the allowed ranges and choices for a specific property are. The ADSpinnaker driver is only 60% of the size of the old ADPointGrey driver because of the simplicity of the SDK. The choices for menu-type properties are built at run-time from the values supported by the camera. For some properties these choices can change dynamically when some other property is changed. EPICS Channel Access does not support callbacks when enum choices change. This means that GUI clients cannot update the choices on an open screen when the driver changes them. For this reason enum records that can change their choices at run-time are displayed on a related display (ADSpinnakerMore) that must be closed and re-opened to see the new choices. The valid range of numeric properties can similarly change dynamically at run-time based on changes in other properties. These changes will be immediately reflected the DRVL, DRVH, LOPR, and HOPR fields of the records. However, EPICS Channel Access does not support callbacks when graphics display limits change, so EPICS GUI clients do not update slider widget limits when HOPR and LOPR change. For this reason numeric records that can change their choices at run-time are also displayed on ADSpinnakerMore that must be closed and re-opened to have the slider limits update.

Standard areaDetector parameters

The following table describes how the Point Grey driver implements some of the standard driver parameters.

Implementation of Parameters in asynNDArrayDriver.h and ADDriver.h, and EPICS Record Definitions in ADBase.template and NDFile.template
Parameter index variable EPICS record name Description
ADTriggerMode $(P)$(R)TriggerMode The choices for the Point Grey are:
  • "Internal". The timing is internal to the detector.
  • "Ext. standard". Each external trigger pulse starts the next image. The exposure time is controlled internally by the AcquireTime record.
  • "Bulb". The rising edge of the external trigger signal starts the next image. The detector continues to acquire while the external signal is high, and then reads out the detector on the falling edge of the external trigger signal.
  • "Skip frames". One external trigger pulse starts an image and then the next N external trigger signals are ignored. The SkipFrames record defines N.
  • "Multi-exposure". One external trigger pulse starts an image and then the next N-1 external trigger signals cause an additional exposure into the same image. The image is read out after trigger N. The NumExposures record defines N.
  • "Multi-exposure bulb". A combination of bulb and multi-exposure modes above. N exposures are accumulated into an image before it is read out. The time in the logic high state determines the acquire time for each exposure. The NumExposures record defines N.
  • "Low-smear". Smear reduction works by increasing the speed of the vertical clock near the end of the integration cycle. See the Technical Reference Manual for the camera for more information.
  • "Multi-shot". A single external trigger causes N images to be acquired. The NumImages record defines N. NumImages is limited to a maximum 255 in this mode.
Note that the minimum time between external trigger pulses is no more than the maximum value of FrameRate in the current mode, and may be less for a particular camera. Note also that not all cameras support all TriggerModes. The TriggerMode enum string choices are only those supported for the camera in use.
ADTemperatureActual $(P)$(R)TemperatureActual The readback of the temperature.
ADNumImages $(P)$(R)NumImages Controls the number of images to acquire. When TriggerMode=Internal this is handled in software. When TriggerMode=Multi-shot it is handled in the camera firmware.
ADNumExposures $(P)$(R)NumExposures Controls the number of exposures per image when TriggerMode="Multi-exposure" or "Multi-exposure bulb".
ADAcquireTime $(P)$(R)AcquireTime Controls the acquisition time per image. This is converted into the ShutterAbsVal control of the SHUTTER property. ShutterAbsVal = AcquireTime*1000., because SHUTTER units are ms.
ADAcquirePeriod $(P)$(R)AcquirePeriod Controls the period between images. This is converted into the FrameRateAbsVal control of the FRAME_RATE property. FrameRateAbsVal = 1./AcquirePeriod.
ADGain $(P)$(R)Gain Controls the analog gain on the camera. This is converted into the GainAbsVal control of the GAIN property. The units are dB.

Point Grey specific driver parameters

The Point Grey driver implements the following parameters in addition to those in asynNDArrayDriver.h and ADDriver.h. The database file is spinnaker.template for all records except the property records, which are in spinnakerProperty.template.

Parameter Definitions in firewireWinDCAM.cpp and EPICS Record Definitions
Parameter index variable asyn interface Access Description drvInfo string EPICS record name EPICS record type
Video mode parameters
PGVideoMode asynInt32 r/w The video mode. All possible values are listed in the Video modes section above. The actual enum choices for this record will only include the video modes supported by the camera in use. PG_VIDEO_MODE $(P)$(R)VideoMode
$(P)$(R)VideoMode_RBV
mbbo
mbbi
PGFormat7Mode asynInt32 r/w The Format7 mode when the camera is in VideoMode=Format7. This is discussed in the Format7 modes section above. The actual enum choices for this record will only include the Format7 modes supported by the camera in use. PG_FORMAT7_MODE $(P)$(R)Format7Mode
$(P)$(R)Format7Mode_RBV
mbbo
mbbi
PGPixelFormat asynInt32 r/w The pixel format when the camera is in VideoMode=Format7. This is discussed in the Pixel formats section above. The actual enum choices for this record will only include the pixel formats supported by the camera in use for the Format7Mode currently selected. PG_PIXEL_FORMAT $(P)$(R)PixelFormat
$(P)$(R)PixelFormat_RBV
mbbo
mbbi
PGConvertPixelFormat asynInt32 r/w The driver allows converting the pixel format from the camera to another pixel format. The pixel formats from the camera that can be converted are:
  • Raw8
  • Raw12
  • Raw16
  • Mono12
The pixel format that these can be converted to are:
  • None: The input pixel format is not converted.
  • Mono8: The input pixel format is converted to Mono8.
  • Raw16: The input pixel format is converted to Raw16. This is useful when the input pixel format is Raw12, since this saves network bandwidth.
  • Mono16: The input pixel format is converted to Mono16. This is useful when the input pixel format is Mono12, since this saves network bandwidth. Note that the FlyCap2 library does not support converting Raw12 to Mono16. The difference between Raw16 and Mono16 is that Mono16 has the sharpness and gamma corrections applied, while Raw16 does not.
  • RGB8: The input format is converted to RGB8. This is useful when the input format is Raw8, for a color camera. In this case Bayer color is sent on the network, reducing the bandwidth requirement by a factor of 3.
  • RGB16: The input format is converted to RGB16. This is useful when the input format is Raw16, for a color camera. In this case Bayer color is sent on the network, reducing the bandwidth requirement by a factor of 3.
PG_CONVERT_PIXEL_FORMAT $(P)$(R)ConvertPixelFormat
$(P)$(R)ConvertPixelFormat_RBV
mbbo
mbbi
PGFrameRate asynInt32 r/w The frame rate choice when the VideoMode is not Format7. This is discussed in the Frame rates section above. The actual enum choices for this record will only include the frame rates supported by the camera in use for the VideoMode currently selected. PG_FRAME_RATE $(P)$(R)FrameRate
$(P)$(R)FrameRate_RBV
mbbo
mbbi
BinningMode asynInt32 r/w The binning mode for GigE cameras. The allowed values are camera-specific. PG_BINNING_MODE $(P)$(R)BinningMode
$(P)$(R)BinningMode_RBV
mbbo
mbbi
Property parameters
These parameters apply to each of the 18 Point Grey properties discussed in the Properties section above.
The $(PROPERTY) macro in this table is the EPICS record base name listed in that section.
These records are defined in spinnakerProperty.template.
PGPropertyAvail asynInt32 r/o A flag indicating if the property is available. PG_PROP_AVAIL $(P)$(R)$(PROPERTY)Avail bi
PGPropertyOnOffAvail asynInt32 r/o A flag indicating if the property supports turning on and off. PG_PROP_ON_OFF_AVAIL $(P)$(R)$(PROPERTY)OnOffAvail bi
PGPropertyOnOff asynInt32 r/w Controls whether the property is on or off. PG_PROP_ON_OFF $(P)$(R)$(PROPERTY)OnOff
$(P)$(R)$(PROPERTY)OnOff_RBV
bo
bi
PGPropertyOnePushAvail asynInt32 r/o A flag indicating if the property supports setting once (called One Push). This is typically used for setting things like the gain or shutter time automatically once. PG_PROP_ONE_PUSH_AVAIL $(P)$(R)$(PROPERTY)OnePushAvail bi
PGPropertyOnePush asynInt32 r/w Processing this record causes a one-push setting of the property. PG_PROP_ONE_PUSH $(P)$(R)$(PROPERTY)OnePush bo
PGPropertyAbsAvail asynInt32 r/o A flag indicating if the property supports absolute (floating point) control. PG_PROP_ABS_AVAIL $(P)$(R)$(PROPERTY)AbsAvail bi
PGPropertyAutoAvail asynInt32 r/o A flag indicating if the property supports automatic control. PG_PROP_AUTO_AVAIL $(P)$(R)$(PROPERTY)AutoAvail bi
PGPropertyManAvail asynInt32 r/o A flag indicating if the property supports manual control. PG_PROP_MAN_AVAIL $(P)$(R)$(PROPERTY)ManAvail bi
PGPropertyAutoMode asynInt32 r/w Controls whether the property is manually controlled or automatically controlled. PG_PROP_AUTO_MODE $(P)$(R)$(PROPERTY)AutoMode
$(P)$(R)$(PROPERTY)AutoMode_RBV
bo
bi
PGPropertyAbsAvail asynInt32 r/o A flag indicating if the property supports absolute (floating point) control. PG_PROP_ABS_AVAIL $(P)$(R)$(PROPERTY)AbsAvail bi
PGPropertyAbsMode asynInt32 r/w Controls whether the property is controlled in integer device units or floating point absolute units. PG_PROP_ABS_MODE $(P)$(R)$(PROPERTY)AbsMode
$(P)$(R)$(PROPERTY)AbsMode_RBV
bo
bi
PGPropertyValue asynInt32 r/w The value of the property in integer device units. This controls the ValueA field of the property, which is the only integer field used for all properties except WHITE_BALANCE. PG_PROP_VAL $(P)$(R)$(PROPERTY)Val
$(P)$(R)$(PROPERTY)Val_RBV
ao
ai
PGPropertyValueB asynInt32 r/w The value of the property in integer device units. This controls the ValueB field of the property, which is only used to control the Blue value of WHITE_BALANCE. PG_PROP_VAL_B $(P)$(R)$(PROPERTY)ValB
$(P)$(R)$(PROPERTY)ValB_RBV
ao
ai
PGPropertyValueMin asynInt32 r/o The minimum value of the property in device units. This is used to control the LOPR and DRVL fields of the $(P)$(R)$(PROPERTY)Val record. PG_PROP_VAL_MIN $(P)$(R)$(PROPERTY)ValMin ai
PGPropertyValueMax asynInt32 r/o The maximum value of the property in device units. This is used to control the HOPR and DRVH fields of the $(P)$(R)$(PROPERTY)Val record. PG_PROP_VAL_MAX $(P)$(R)$(PROPERTY)ValMax ai
PGPropertyValueAbs asynFloat64 r/w The value of the property in floating point absolute units. PG_PROP_VAL_ABS $(P)$(R)$(PROPERTY)ValAbs
$(P)$(R)$(PROPERTY)ValAbs_RBV
ao
ai
PGPropertyValueAbsMin asynFloat64 r/o The minimum value of the property in absolute units. This is used to control the LOPR and DRVL fields of the $(P)$(R)$(PROPERTY)ValAbs record. PG_PROP_VAL_ABS_MIN $(P)$(R)$(PROPERTY)ValAbsMin ai
PGPropertyValueAbsMax asynFloat64 r/o The maximum value of the property in absolute units. This is used to control the HOPR and DRVH fields of the $(P)$(R)$(PROPERTY)ValAbs record. PG_PROP_VAL_ABS_MAX $(P)$(R)$(PROPERTY)ValAbsMax ai
GigE Property parameters
These parameters apply to each of the 4 Point Grey GigE properties discussed in the GigE Properties section above.
The $(PROPERTY) macro in this table is the EPICS record base name listed in that section.
These records are defined in spinnakerProperty.template.
PGPropertyValue asynInt32 r/w The value of the property in integer device units. This controls the ValueA field of the property, which is the only integer field used for all properties except WHITE_BALANCE. PG_PROP_VAL $(P)$(R)$(PROPERTY)Val
$(P)$(R)$(PROPERTY)Val_RBV
ao
ai
PGPropertyValueMin asynInt32 r/o The minimum value of the property in device units. This is used to control the LOPR and DRVL fields of the $(P)$(R)$(PROPERTY)Val record. PG_PROP_VAL_MIN $(P)$(R)$(PROPERTY)ValMin ai
PGPropertyValueMax asynInt32 r/o The maximum value of the property in device units. This is used to control the HOPR and DRVH fields of the $(P)$(R)$(PROPERTY)Val record. PG_PROP_VAL_MAX $(P)$(R)$(PROPERTY)ValMax ai
Trigger parameters
PGTriggerSource asynInt32 r/w The trigger source signal. Choices are "GPIO_0", GPIO_1","GPIO_2", and "GPIO_3", which selects one of 4 GPIO pins on the camera. However, not all choices may be available on a specific camera, and the enum choices will only be the trigger sources actually supported on the camera in use. PG_TRIGGER_SOURCE $(P)$(R)TriggerSource
$(P)$(R)TriggerSource_RBV
mbbo
mbbi
PGTriggerPolarity asynInt32 r/w The trigger polarity. Choices are "Low", and "High". PG_TRIGGER_POLARITY $(P)$(R)TriggerPolarity
$(P)$(R)TriggerPolarity_RBV
bo
bi
PGSoftwareTrigger asynInt32 r/w Processing this record causes the driver to issue a software trigger. PG_SOFTWARE_TRIGGER $(P)$(R)SoftwareTrigger bo
PGSkipFrames asynInt32 r/w The number of frames to skip when TriggerMode="Skip frames". PG_SKIP_FRAMES $(P)$(R)SkipFrames
$(P)$(R)SkipFrames_RBV
longout
longin
Strobe parameters
PGStrobeSource asynInt32 r/w The strobe output signal. Choices are "GPIO_0", GPIO_1","GPIO_2", and "GPIO_3", which selects one of 4 GPIO pins on the camera. However, not all choices may be available on a specific camera, and the enum choices will only be the strobe sources actually supported on the camera in use. PG_STROBE_SOURCE $(P)$(R)StrobeSource
$(P)$(R)StrobeSource_RBV
mbbo
mbbi
PGStrobeEnable asynInt32 r/w Enables the strobe output. Choices are "Disable", and "Enable". PG_STROBE_ENABLE $(P)$(R)StrobeEnable
$(P)$(R)StrobeEnable_RBV
bo
bi
PGStrobePolarity asynInt32 r/w The strobe polarity. Choices are "Low", and "High". PG_STROBE_POLARITY $(P)$(R)StrobePolarity
$(P)$(R)StrobePolarity_RBV
bo
bi
PGStrobeDelay asynFloat64 r/w The delay of the strobe signal relative to the start of acquisition. PG_STROBE_DELAY $(P)$(R)StrobeDelay
$(P)$(R)StrobeDelay_RBV
ao
ai
PGStrobeDuration asynFloat64 r/w The duration of the strobe signal. If zero then the strobe output is asserted during the image aquisition time. PG_STROBE_DURATION $(P)$(R)StrobeDuration
$(P)$(R)StrobeDuration_RBV
ao
ai
Bandwidth control parameters
PGMaxPacketSize asynInt32 r/o The maximum packet size. This depends on the current acquisition settings for Firewire and USB cameras. For GigE cameras this is determined by calling DiscoverGigEPacketSize at startup, which should return the maximum Ethernet packet size supported between the camera and the IOC. However, this sometimes returns 9000 (jumbo packets) when jumbo packets are not in fact supported. In this case the user should manually set PacketSize to 1440 or image acquisition will fail. PG_MAX_PACKET_SIZE $(P)$(R)MaxPacketSize longin
PGPacketSize asynInt32 r/w The packet size to use. This is used to control the maximum bandwidth, and hence maximum frame rate, on Firewire and USB cameras. For GigE cameras it should be set to the largest packet size supported on the Ethernet connection between the camera and IOC. It should be set to 1440 for connections that do not support jumbo packets, and as large as 9000 for connections that do support jumbo packets. If PacketSize is set to 0 then the driver will use the current value of MaxPacketSize. PG_PACKET_SIZE $(P)$(R)PacketSize
$(P)$(R)PacketSize_RBV
ao
ai
PGPacketSizeActual asynInt32 r/o The actual packet size being used. PG_PACKET_SIZE_ACTUAL $(P)$(R)PacketSizeActual longin
PGPacketDelay asynInt32 r/w The packet delay to use in microseconds. This is used to control the maximum bandwidth, and hence maximum frame rate, on GigE cameras. It is not used for Firewire or USB cameras. The default is 400 microseconds. If the number of CorruptFrames is large then this can be increased, for example to 1000. This will reduce the maximum frame rate but can significantly reduce the number of CorruptFrames. PG_PACKET_DELAY $(P)$(R)PacketDelay
$(P)$(R)PacketDelay_RBV
longout
longin
PGPacketDelayActual asynInt32 r/o The actual packet delay being used. PG_PACKET_DELAY_ACTUAL $(P)$(R)PacketDelayActual longin
PGBandwidth asynFloat64 r/o The calculated bandidth in MB/s. This is computed from the image size and the frame rate. PG_BANDWIDTH $(P)$(R)Bandwidth ai
Timestamp parameters
PGTimeStampMode asynInt32 r/w The timestamp mode. Controls the value of the NDArray,.timeStamp value. Choices are:
  • Camera: The time from the camera is used.
  • EPICS: The EPICS time is used
  • Hybrid: The EPICS time when the camera started is combined with the time stamp from the camera.
PG_TIME_STAMP_MODE $(P)$(R)TimeStampMode
$(P)$(R)TimeStampMode_RBV
mbbo
mbbi
Camera statistics
PGCorruptFrames asynInt32 r/o The number of corrupt frames. The Point Grey SDK resets this to 0 each time acquisition is started. PG_CORRUPT_FRAMES $(P)$(R)CorruptFrames_RBV longin
PGDroppedFrames asynInt32 r/o The number of dropped frames. The Point Grey SDK resets this to 0 each time acquisition is started. PG_DROPPED_FRAMES $(P)$(R)DroppedFrames_RBV longin
PGDriverDropped asynInt32 r/o The number of frames dropped by the driver. The Point Grey SDK resets this to 0 each time acquisition is started. PG_DRIVER_DROPPED $(P)$(R)DriverDropped_RBV longin
PGTransmitFailed asynInt32 r/o The number of time transmission failed. The Point Grey SDK resets this to 0 each time acquisition is started. PG_TRANSMIT_FAILED $(P)$(R)TransmitFailed_RBV longin

Configuration

The ADSpinnaker driver is created with the spinnakerConfig command, either from C/C++ or from the EPICS IOC shell.

spinnakerConfig(const char *portName, const char* cameraId, int traceMask, int memoryChannel,
              int maxBuffers, size_t maxMemory, 
              int priority, int stackSize)
  

The cameraId parameter can either be an index of the camera in the list of available cameras (e.g. 0 if there is only a single Point Grey camera available) or the serial number of the camera to use. For additional details on the meaning of the parameters to this function refer to the detailed documentation on the spinnakerConfig function in the spinnaker.cpp documentation and in the documentation for the constructor for the spinnaker class.

The traceMask can be set to a value > 1 to enable asynTrace debugging during initialization, before the value can be set from the IOC shell or via the asynRecord. Set this to 0x21 to enable ASYN_TRACE_WARNING, which will trace all calls to the Point Grey FlyCap2 library.

The memoryChannel can be set to a value > 0 to load the initial camera parameters from non-volatile memory in the camera. Setting memoryChannel to N loads from memoryChannel N-1, i.e. 1 loads memory channel 0. There is currently a problem with Linux and for BlackFly GigE cameras. If the IOC is run a second time after it has been used to acquire any images the driver loses communication with the camera. The problem appears to be that there is a corrupt setting in the camera, which causes it to malfunction the next time the program is run. Setting memoryChannel 1 will work around this problem by replacing the settings in the camera with a default set. Since the EPICS IOC sets nearly all the camera settings to save/restore values at startup anyway, this is not a serious limitation.

There an example IOC boot directory and startup script (iocBoot/iocSpinnaker/st.cmd) provided with areaDetector.

MEDM screens

The following show the MEDM screens that are used to control the Point Grey cameras.

spinnaker.adl is the main screen used to control Point Grey cameras.

spinnaker.adl

ADSpinnaker.png

spinnakerProperties.adl is the screen used to control the properties of Point Grey cameras. Note that some of these properties, such as Shutter, FrameRate, and Gain can also be controlled by standard areaDetector records, like AcquireTime, AcquirePeriod, and Gain. The widgets on the medm screen are hidden if the corresponding feature is not available.

spinnakerProperties.adl for a BlackFly GigE color camera in RGB mode

spinnakerProperties_BlackFly.png

spinnakerProperties.adl for a Grasshopper3 monochrome camera

spinnakerProperties_Grasshopper3.png

spinnakerFrameRate.adl is the screen used to control the frame rate in standard video modes. This is a separate screen because the valid enum strings for the Framerate record can change when the standard video mode is changed. When that is changed it is necessary to close this screen and re-open it in order for the new menus to be displayed. This is a limitation of the EPICS Channel Access which does not send monitor events for changes in enum fields. Note that the readback of the FrameRate on the main spinnaker.adl screen can also be incorrect, so it may be necessary to close and re-open that main screen as well.

spinnakerFrameRate.adl

spinnakerFrameRate.png

spinnakerPixelFormat.adl is the screen used to control the pixel format in Format7 mode. This is a separate screen because the valid enum strings for the PixelFormat can change when the Format7 mode is changed. When that is changed it is necessary to close this screen and re-open it in order for the new menus to be displayed. This is a limitation of the EPICS Channel Access which does not send monitor events for changes in enum fields. Note that the readback of the PixelFormat on the main spinnaker.adl screen can also be incorrect, so it may be necessary to close and re-open that main screen as well.

spinnakerPixelFormat.adl

spinnakerPixelFormat.png