THine Camera Development Kit (CDK) Speeds Up Time to Market for Medical Device Start-up

Russell Lemburg, Product Engineer, Evergaze LLC

Evergaze LLC is a Texas start-up that develops and manufactures seeBOOST, a proprietary medical assistive device for persons with central vision loss due to Age-related Macular Degeneration (AMD) or diabetes. seeBOOST overcomes the significant drawbacks of currently marketed assistive devices, whether head- or spectacle-mounted, handheld or implanted. These devices falls short of addressing the primary needs of central vision loss.

Figure 1 First-Generation seeBOOST device for Central Vision Loss

In particular, these head-mounted devices are heavy, cover the eyes or face completely, and block good remaining peripheral vision not directly affected by central vision loss. From a technical perspective, small and low-power components with high degrees of low-level software capability are hard to come by for custom AR/VR applications for specific applications.

As image sensors have increased in resolution and bandwidth requirements, and ISPs have generally moved outside of the sensor and into SoCs or external processors, the need for high performance and low-power ISPs for camera applications has increased. Using SRAM-based FPGAs generally result in higher static power consumption, and an ASIC or full-custom has added risk in terms of time-to-market and initial cost to tapeout. THine’s ISP offerings mitigate every issue here, in that they offer:

• Very small IC package size options for tight package constraints (3.9 × 3.9 ×0.62 mm)

• Very low power operation (an order of magnitude less than other options, i.e. Qualcomm SoCs). This improves the overall thermal profile and battery requirements for portable applications.

• A series of easy-to-use development tools that offer sophisticated tuning capabilities and real-time adjustment.

• Top-notch customer support and integration/development assistance. THine’s approach is very much “get the customer to market”. Even with small startups, the technical support in place is second to none.

THine’s ISP solutions are standalone ICs capable of very low-latency camera-to-display systems with low power consumption, without the need for external DDR. For designs that require well-known image processing operations at the sensor rate, THine’s ISP offers a balanced tradeoff between full frame-buffer approach (with additional latency) and a stream-processing approach. This approach requires less physical components, a less complex PCB, lower BOM cost, and lower power with less overall latency.

► Camera Development Kit (CDK)

Figure 2 THine’s CDK for custom application.

TheThe Camera Development Kit is an integrated solution for developing a full end-solution usingHine’s ISP. The CDK is comprised of a base board, containing interfacing and translation for video output. An ISP board plugs into the base board, which enables the connection of various sensor modules (sensor + lens + actuator). This board enables direct connectivity to a development system for programming, debugging and visualization of the ISP output data stream.

THine’s Tuning Tool (TTT) is the CDK’s software for programming the embedded microprocessor in the ISP. This software enables the customer to:

• Make real-time adjustments to the sensor image, including AE/AWB/IQ, tone maps, color configuration, shading/lens correction, AWB, and timing adjustment while the sensor is streaming.

• Debugging tools support RAW v. Normal output, bayer pattern re-ordering, image and video capture, and I2C/SPI communication to the sensor and ISP.

Figure 3 Driving Mode Configuration Tool

The Driving Mode Configurator (DMC) enables very specific timing control of both the sensor and ISP, enabling operations like image scaling and global exposure. With respect to image scaling, the camera sensor timing and region-of-interest (ROI) are properly matched to the scale factor and sensor’s subsampling modes (binning/skipping), which gives much sharper results at higher scale factors.

The sensor configurations from the DMC tool can be exported, and integrated into the customer’s application very easily. For example, a customer may want to have a “low-light” and “high frame rate” mode. Both of these configurations can be easily created to drive the sensor and ISP with the correct frame and line timing, and exported into the software development environment.

Figure 4 Gain based tuning of ISP parameters

Gain-Based Tuning
TTT offers a gain-based tuning approach across multiple ISP functions. For example, noise reduction can be increased nonlinearly as a function of gain. Multiple tables of gain vs. ISP function can be employed. These gain based settings include sharpness, color, and noise-reduction.

Color Configuration
The AWB/IQ Color Configurator tool combined with a ColorChecker and appropriate lighting enables customers to perform accurate color configuration in a matter of minutes.

Within the USBCamViewer tool, a Macbeth Analyzer enables software-driven color matching based on the color temperature of the light source.

Figure 5 Real-time Histogram and RGB Level viewers

A few other useful tools available within the USBCamViewer tool are real-time RGB histograms within selectable image regions, and a level scope. The CDK can also be connected directly to a monitor through DVI.

► CDK Software Development
THine’s ISP uses the well-known CodeWarrior development environment, based on the Eclipse IDE.

From this environment, a straightforward I2C memory-mapped approach for ISP configuration can be designed. A get/put methodology for handling commands from a host MCU is employed, where a base address and data transmitted from a host processor can control all of the internal functions of the ISP, or the internal processor can be used as the sole processor. The embedded processor is capable of computation at the frame rate, which enables responsive image processing adjustments (i.e. detect image change, change timing parameters, turn on/off ISP features).

► Final Thoughts
THine’s ISP provide a high-speed pipelined and low-power approach to interfacing a variety of CMOS image sensors for custom applications. It offers the smallest package size of any ISP, with a 3.9 × 2.9 × 0.62mm WLCSP option.

The functions provided include HDR, image stabilization, black level correction, dust/defect pixel correction, lens shading, demosaicing, adaptive and fully programmable noise-reduction, edge enhancement, color and gamma correction (all with gain-based parameterization), continuous, multi-point auto-focus, AWB, flicker cancellation, JPEG encoding, digital zoom, and a wide variety of special effects. The ISP can also control peripherals including VCMs, Piezoelectrics, and LED/Xenon flash.

A truly one of a kind approach for any customers thinking about real-time image processing without using a full SoC, operating system, external flash and DDR memory, but with the additional benefit of a custom tuning system which provides additional flexibility and shorter time to market.