34th Annual International Conference of the IEEE EMBS San Diego, California USA, 28 August - 1 September, 2012 ASIC Design and Data Communications for the Boston Retinal Prosthesis Douglas B. Shire, Member, IEEE, William Ellersick, Shawn K. Kelly, Member, IEEE, Patrick Doyle, Attila Priplata, William Drohan, Oscar Mendoza, Marcus Gingerich, Bruce McKee, John L. Wyatt, Senior Member, IEEE, and Joseph F. Rizzo III, MD Abstract²We report on the design and testing of a custom application-specific integrated circuit (ASIC) that has been developed as a key component of the Boston retinal prosthesis. This device has been designed for patients who are blind due to age-related macular degeneration or retinitis pigmentosa. Key safety and communication features of the low-power ASIC are described, as are the highly configurable neural stimulation current waveforms that are delivered to its greater than 256 output electrodes. The ASIC was created using an 0.18 micron Si fabrication process utilizing standard 1.8 volt CMOS transistors as well as 20 volt lightly doped drain FETs. The communication system receives frequency-shift keyed inputs at 6.78 MHz from an implanted secondary coil, and transmits data back to the control unit through a lower-bandwidth channel that employs load-shift keying. The design's safety is ensured by on-board electrode voltage monitoring, stimulus charge limits, error checking of data transmitted to the implant, and comprehensive selftest and performance monitoring features. Each stimulus cycle is initiated by a transmitted word with a full 32-bit error check code. Taken together, these features allow researchers to safely and wirelessly tailor retinal stimulation and vision recovery for each patient. I. INTRODUCTION T HE retinal neural network is a part of the brain with well-understood geographic organization; patients have provided detailed feedback on its response to electrical stimuli . Advances in micro-fabrication, electronics, materials, surgical techniques and research have enabled the Boston Retinal Implant group (BRIP) to build an advanced retinal prosthesis to restore vision to patients who have lost Manuscript received March 29, 2012. This work was supported in part by the U.S. Department of Veterans Affairs, Rehabilitation Research and Development Division, through Center of Excellence grant C4266-C. Additional support was provided by NIH grant EY016674 to Prof. Wyatt at MIT, by the 16)¶V VXSSRUW RI WKH &RUQHOO 1DQR6FDOH )DFLOLW\ and Prof. Wyatt, by the Massachusetts Lions Foundation, and by private sources. D. B. Shire, S. K. Kelly, M. D. Gingerich, W. Drohan and B. McKee are with the Boston VA Healthcare System, 150 S. Huntington Ave., Boston, MA 02130 (phone: +1 (607) 339-7085; fax: (607) 255-8601; e-mail: [email protected]). O. Mendoza and J. L. Wyatt are with the Massachusetts Institute of Technology, Cambridge, MA 02139; J. F. Rizzo and A. Priplata are with the Massachusetts Eye and Ear Infirmary, Boston, MA 02114; and P. Doyle is with Harvest Automation, Billerica, MA 01862 USA. U.S. Government work not protected by U.S. copyright their vision due to retinal diseases, especially retinitis pigmentosa and age-related macular degeneration. In recent years, the visual prosthetic community has expanded to more than 20 groups worldwide -. The collective outcome of testing (see, e.g., -) KDV SURYLGHG LPSRUWDQW ³SURRIof-FRQFHSW´ HYLGHQFH LQFOXGLQJ GHPRQVWUDWLRQ RI seemingly safe stimulation thresholds; 2) that severely blind patients can reliably see phosphenes; 3) that modulation of electrical stimulation alters percepts; 4) that object orientation and movement can be detected; and 5) that restoration of vision with an acuity of about 20/1000 is possible, although this often requires head movement to scan letters and considerable time to interpret the visual scene, sometimes tens of seconds. One notable result from Retina Implant AG in Germany was an implant recipient who could read one week after implantation . Collectively, these results surely are encouraging, but fall short of the goal of creating more natural, higher quality vision that would more easily justify the risk of surgery and the cost of the device. The need to achieve higher quality of restored vision has motivated us as well as other teams - to develop advanced devices, which our team chose to do prior to performing long-term human implants. We report on the application specific integrated circuit (ASIC) that forms the heart of our new prosthetic, the wireless means used to communicate data to and from our device, and the means for processing input images gathered from a video camera mounted on glasses worn by the patient. Our results will pave the way toward human trials, and, we hope, a real improvement in quality-of-life for implant recipients. II. ASIC DESIGN AND TESTING A. System Description The core element of our retinal prosthesis is the ASIC that serves as its communication and control center. It is designed to receive information from an external controller, control the electrode drivers, and transmit outbound data. The µUHYHUVH¶ Welemetry includes monitoring of electrode voltage waveform data to ensure the safety of the patient and the long-term integrity of the electrodes, and data to support the debugging of implanted devices or lab experiments. Figure 1 shows a block diagram of the BRI2 ASIC. In the upper left, a bandgap voltage reference (VRF) and an 292 external resistor allow the REF block to distribute constant voltage and constant current references across the 5x5mm System on Chip (SoC). The TMP block generates a current proportional to absolute temperature that is used to sense changes in die temperature. The PWR block regulates onchip power supplies, while the RCV and XMT blocks communicate with the external controller using frequencyshift-keying (FSK) and load-shift-keying (LSK), respectively. A synthesized digital control logic block interfaces between the RCV and XMT blocks and the array of electrode drivers (DRV), as well as the on-chip analog test PXOWLSOH[RUV $767 WKDW IHHG GXDO $'&¶V DQG WKH electrode voltage monitor (DRVMON) that checks for charge buildup on electrodes. An on-chip memory (MEM) FDSWXUHV WKH RXWSXW RI WKH GXDO $'&¶V RU DQ\ RI D ZLGH UDQJH of other on-chip digital signals, with a configurable sample rate and start time. REF IRF vbg VRF izt4u ibg4u adcvcm itmp TMP XMT ATST DAC static sigs vddh gnd vdd! PWR vss! cmp*ref1 RCV states dyn. sigs SMP atst[1:0] MEM ADC ADC MUXHVL rclk rxd DRVMON control ctl[N:0] DRV logic iOut[N:0] txd levels of electrode polarization. While allowing both highcurrent short pulses and low-current long pulses, the ASIC has hardware-enforced charge limits, guided by our prior electrode characterization work . Configuration pins allow the limits to be changed for compatibility with a range of electrode sizes. Each electrode is grounded (shorted to the case) for at least 200 µsec before any stimulus, and its voltage is monitored to ensure that the electrode is fully depolarized, and monitored again after the initial W1 pulse to ensure that electrode polarization is within safe limits (see Figure 2). The VDDH and VSS power supplies prevent excessively large voltages from being driven on electrodes. In addition, the integrated ADCs periodically sample the voltage waveforms on each electrode, sending back detailed measurements to the external controller; this allows open and short circuits to be detected, as well as changes in electrodetissue impedance and responses over time. To reduce bandwidth requirements, an efficient message organization is used. Forward and reverse channels operate simultaneously while stimuli are ongoing. In fact, restricting the stimuli to charge-balanced waveforms for safety also reduces the degrees of freedom, and thus the number of bits needed to describe the waveforms. Figure 2 shows a first command received from the external controller (RxCommand1), which initiates a set of stimulus pulses while the subsequent RxCommand2 is being received. The commands specify the time width W1 and the sign of current for the initial pulse, along with the inter-phase width Wip and the complementary pulse width W2. For charge balance, I2 = I1 * (W1/W2). Fig. 1. BRI2 ASIC block diagram. RxCommand1 TABLE 1 RxCommand2 BRI2 Implant ASIC Technical Specifications W1 Ielectrode Power: 6.78 MHz carrier; ~30 mW transmitted Data: 565 Kbps inbound, 70 Kbps outbound Electrode Current: 0 ± 127 A in 1 A steps Stimulus Pulse width: 17.7 ± 4500 sec; up to +/- 8 V Wip W2 I1 I2 Fig.2. Electrode stimulus control and waveform. Safety features include stimulus charge limits, error checking on data transmission, comprehensive self-test and performance monitoring, and configuration pins to change operating modes or to lock out settings that might somehow cause harm. This allows debugging and optimization of performance in the lab, and management of risk. A robust ACK-only protocol with strong error checking is used to communicate with the implant ASIC. Each message from the external controller can result in a set of stimulus pulses on a subset of the electrodes, and generates a corresponding response from the implant ASIC with chip and electrode status, and any other requested data. Radio communication errors are nearly eliminated by the use of 32-bit CRCs, reducing the probability of accepting a corrupted message to 2-10. No individual message can cause harm, and messages with a bad CRC merely result in missing stimulus pulses. Several features also work to ensure operation with safe Any subset of electrodes can be driven in a given stimulus cycle, and electrodes can be configured as sources of current or as local current sinks (returns) to provide current steering capabilities. There is considerable flexibility in pulse repetition rate, and a 60 Hz refresh of all electrodes can be wirelessly commanded as needed. Power consumption is minimized in a number of ways, which ultimately limits the RF energy the recipient must be exposed to while the implant is active. Analog bias currents are typically 1µA, and are distributed to local current mirrors shared by the electrode drivers, which consume no power during the majority of the time that they are not enabled; brief power-up transients are charge-matched to first order between positive and negative voltage sources. The ability to connect almost any analog node, including all electrodes, to on-FKLS $'&¶V IRU ZDYHIRUP WUDQVPLVVLRQ back to the external controller, or to an on-chip DAC, enabled testing of much of the analog circuitry. A similar ability to capture or drive many digital signals, along with a 293 built-in scan test, also allowed faults to be detected. A micrograph of the BRI2 retinal implant die is shown in Figure 3. Part of the electrode driver array is in the upper two thirds of the image, and centralized bias circuitry is at the lower right; both are under a power and signal grid in the transmitted power dropped off by 10% . A resonant capacitor was included on a small board inside the titanium enclosure, along with Schottky diodes and capacitors to halfwave rectify a positive and negative supply relative to GND. Since our charge-balanced stimulus drew equal currents from VDDH and VSS, the dual half-wave rectification was efficient, and minimized discrete components within the package. Circuitry in the ASIC regulated VDDH and VSS to minimize power and provided accurate stimuli. The implant power supplies can run in two modes, +/- 8V, and +/- 4V. These relatively high voltages are dropped mostly across tissue; this is acceptable because of the low field due to the large distance to the GND return. EXTXMT VDDH GND Fig. 3. Partial, wire-bonded die photo showing some of the >256 electrode drivers, control logic, and analog circuits. VSS Fig. 5. BRI2 ASIC radio and power system block diagram. WKLFN WRS PHWDO OD\HU 7KH GLH ZDV IDEULFDWHG LQ ,%0¶V KLJKvoltage 0.18µm CMOS process, with 20V LD MOSFETs. B. Verification: ASIC, Power and Data Transmission Bench testing of the ASIC using series RC electrode models was performed (see Fig. 4). The lower trace in Figure 4 represents an end-of-stimulus pulse, upon which the electrode is shorted to the current return (GND) that is connected to the platinum-coated, laser-welded Ti package. Fig.6. Secondary coil, VDDH and VSS with simultaneous FSK and LSK. Fig. 4. Upper trace: typical output of the HD retinal micro-stimulator ASIC when powering an electrode modeled as a series RC circuit. Lower trace: an end-of-stimulus pulse, upon which the electrode is shorted to ground. Figure 5 shows the external transmitter (EXTXMT) with its Class E power amplifier connected to a resonant capacitance in parallel with a 7µH primary coil (mounted in glasses that patients will wear). The 4.4µH secondary coil, implanted around the cornea, is approximately 1cm from the glasses, with coupling factor k ~ 0.14. Previous testing by our group showed that at a typical maximum angular displacement from normal alignment of the primary and secondary coils of 15 degrees due to eye movement, the Fig. 7. Wireless data transmission to the retinal implant ASIC. Figure 6 shows the voltages on the secondary coil, VDDH and VSS supplies during power-up of the BRI2 ASIC. The high frequency AC is occasionally attenuated at the ASIC to transmit data back to the external controller using load-shiftkeying (LSK). Somewhat counter-intuitively, LSK that attenuates the secondary coil voltage resulted in a larger signal at the primary coil (in the external glasses), as less 294 power is drawn from the coupled resonant system. Different attenuation strengths are shown, which allow for optimization of data and power transmission. In addition, the power supplies can be seen to be charging up from +/-4V on the left to +/-4.2V on the right, as power is drawn from the secondary coil via the Schottky diodes. A final test was to transmit data to the chip using the RF transmitter. Data reception is demonstrated in Fig. 7, in which the top trace represents WKH FKLS¶V VDPSOLQJ FORFN, the middle waveform represents the received data (which is valid at the rising edge of the sampling clock), and the bottom trace represents the data fed to the transmitter board. C. Video Image Pre-Processing We are currently developing image processing software for the retinal implant and the user interface that patients and clinicians will use together to adjust video pre-processing to achieve the best visual percepts. Images are captured by a glasses-mounted camera at 30 frames/sec and then sent to the external controller, where they are loaded into memory as a 640 x 480 color (RGB) image. They are then converted to grayscale (7-bit) and sent through a series of user-defined filters, chosen e.g. from contrast enhancement, edge extraction, blurring, and thresholding. After filtering, the image is down-sampled to >256 points, one for each electrode. 7KH FDPHUD¶V automatic gain control compensates for varying ambient light levels. The choice of which algorithms to use and what parameters to set for each filter is likely to vary across patients, across different lighting environments, and may also depend on the task that the user is performing. Figure 8 shows an illustration of what video processing might look like. The raw image is sent through a In the portable version of our device, we will implement these algorithms using a dedicated DSP to achieve sufficient computation speed in the embedded system. We anticipate that optimized patient input for image processing algorithms (and stimulation parameters, within safe limits) will be determined in an iterative fashion. III. CONCLUSIONS AND FUTURE WORK We have designed and extensively bench-tested a highly configurable, high-density neuro-stimulator ASIC in both wired and wireless configurations. This >256 channel device is appropriate for chronic implantation with our proven, minimally-invasive sub-retinal surgical implantation techniques, and its sophisticated LSK reverse telemetry features will enable optimization of stimuli for each patient. Extensive safety features have been implemented in this chip; driving software, GUI development, and image processing algorithms for the external system are under development, as we prepare for pilot human trials of our retinal prosthesis. ACKNOWLEDGMENT The authors thank W. Hansford and MOSIS, M. Lugin and M. Segien for custom layout work, G. Galanek and J. Dumser for administrative assistance, the Cornell Nanofabrication staff, and S. Behan for his engineering expertise. REFERENCES      FILTER SELECTIONS: Blurring Edge Extraction Thresholding Contrast Enhancement Zoom In Rotate Image Linear (e.g., low-pass)  Fig. 8. Video filtering and down-sampling for the retinal implant.  thresholding filter and then down-sampled, where each point is spatially aligned with the position of an electrode. Using our user interface, each filter can be turned on or off, and its parameters can be adjusted. The white box in the top left image of Fig. 8 represents a zoom window, allowing the user to zoom in or out on a scene. Currently, the image processing algorithms are implemented on a personal computer running C++/µOpenCV¶. Depending on the extent of the image processing, we achieve frame rates from 5 to 20 frames/sec.   295 J.F. Rizzo III, J. Wyatt, M. Humayun, E. de Juan, W. Liu, A. Chow, R. Eckmiller, E. Zrenner, T. Yagi, DQG * $EUDPV ³Retinal prosthesis: an encouraging first decaGH ZLWK PDMRU FKDOOHQJHV DKHDG´ Ophthalmol., vol. 108, 2008, pp. 13±14. J. I. Loewenstein, S. R. Montezuma, and J. F. Rizzo III, ³Outer retinal degeneration: an electronic retinal prosthesis as a treatment strategy´ Arch. Ophthalmol., vol. 122, 2004, pp. 587±596. J. D. Weiland, W. Liu, DQG 0 6 +XPD\XQ ³Retinal prosthesis´ Ann. Rev. Biomed. Eng., vol. 7, 2005, pp. 361±401. N. Alteheld, G. Roessler, DQG 3 :DOWHU ³7RZDUGV WKH ELRQLF H\H - the retina implant: surgical, opthalmological and histopathological perspectives ´ Acta Neurochir. Suppl., vol. 97, 2007, pp. 487±493. D. B. Shire, S. K. Kelly, J. Chen, P. Doyle, M.D. Gingerich, S. F. Cogan, W. A. Drohan, O. Mendoza, L. Theogarajan, J. L. Wyatt, and J. F. Rizzo ,,, ³Development and implantation of a minimally invasive wireless subretinal neurostimulator ´ IEEE Trans. Biomed. Eng., vol. 56, 2009, pp. 2502±2511. E. Zrenner, K. U. Bartz-Schmidt, H. Benav, D. Besch, A. Bruckmann, V.-P. Gabel, F. Gekeler, U. Greppmaier, A. Harscher, S. Kibbel, J. Koch, J. Kusnyerik, T. Peters, K. Stingl, H. Sachs, A. Stett, P. Szurman, B. Wilhelm, DQG 5 :LONH ³Subretinal electronic chips allow blind patients to read letters and combinH WKHP WR ZRUGV ´ 3URF Biol. Sci., vol. 278, 2011, pp. 1489-1497. K. Chen, Z. Yang, L. Hoang, J. Weiland, M. Humayun, and W. Liu, ³$Q LQWHJUDWHG 256-channel epiretinal prosthesis," IEEE J. Solid-State Circuits, vol. 45, 2010, pp. 1946-1956. N. Tran, E. Skafidas, J. Yang, S. Bai, M. Fu, D. Ng, M. Halpern, and , 0DUHHOV ³$ SURWRW\SH 64-electrode stimulator in 65nm CMOS process towards a high density epi-retinal prosthesis ´ LQ &RQI 3URF IEEE Eng. Med. Biol. Soc., 2011, pp. 6729-6732. S.K. Kelly, P. Doyle, A. Priplata, O. Mendoza, and J.L. Wyatt, "Optimal primary coil size for wireless power telemetry to medical implants," in Conf. Proc. IEEE ISABEL Int. Symp. on Appl. Sci. in Biomed. and Comm. Tech., 2010.