Realization and opto-electronic Characterization of linear Self-Reset Pixel Cells for a high dynamic CMOS Image Sensor

Conventional CMOS image sensors with a linear transfer characteristic only have a limited dynamic range (DR) of about 60–70 dB. To extend the dynamic range considerably, the already successfully demonstrated concept of a linear self-reset pixel was employed in this work. With the self-reset concept the limit of the maximum analyzable photo generated charge (Q_max) during the exposure time is extended to a multiple of the saturation charge of the photo diode (Q_sat) by asynchronous self-resets of the photo diode. Additionally, the remaining charge at the end of the exposure time is evaluated to increase the resolution of the opto-electronic conversion. Thus we achieved pixels with a DR of more than 120 dB combined with an improved low light sensitivity using a pinned photodiode.

This paper focuses on two topics: One is the realization and opto-electronic characterization of further self-reset pixel cells for an experimental optimization of the functionality with respect to linearity and high signal-to-noise ratio. The second one is the assembly and digital readout of a cluster structure composed of 16 × 16 pixel matrix on a CMOS test chip. One constraint for later usage of the pixel cells in a high resolution (> VGA) image sensor is the required layout area of the individual circuit blocks. For the cluster structure a size of 20 × 20 µm² for the analog part of the pixel containing the photo diode and the other analog circuit blocks, the comparator and the signal shaping, was desired. The circuit design and layout work included several variants of the pinned photo diode with floating diffusion (FD) readout node, which is also used for analog voltage storage, and different control transistors. Further for the comparator a telescopic differential amplifier with high gain was implemented as well as peripheral 10 bit counter/shift register as static and dynamic versions. Test chips have been fabricated in an advanced 0.18 µm CMOS technology for optical sensors with low leakage currents.

The sensor chips have been evaluated with a specifically developed test setup which gives the flexibility to arbitrarily generate the digital and analog control signals in terms of timing and voltage levels. Based on this, the number of asynchronous self-resets could be read out from the counters of the pixel cells as coarse values. The remaining charge at the end of the integration time was digitized using a ramp analog to digital conversion and could be read out as fine values. An opto-electronic characterization with adjustable illumination from 0 lx to 13 klx was done to measure and analyze the opto-electronic conversion function (OECF) and the noise of six different self-reset pixel cells having the high-gain differential amplifier as comparator. Finally the coarse values of two implemented 16 × 16 pixel clusters could be read out as a mini camera using a lens for optical image formation.

1 Introduction

Applications in the fields of e.g. industrial image processing, image based robotic control systems and autonomous driving require high dynamic range image sensors. This high dynamic range is needed for robust image capturing and subsequent image processing. A further requirement from measuring point of view is that the image sensor does not only have a linear high dynamic range but also a high signal-to-noise ratio.

The light sensitivity of conventional linear CMOS image sensors has a dynamic range (DR) of about 60 dB (El Gamal and Eltoukhy, 2005) to 70 dB (Seo et al., 2014) only. Techniques to achieve a wide dynamic range (WDR) combine captures of the scene with different exposure times for high and low illumination respectively, like multiple-captures of images or dual-/multi-sampling (Kim et al., 2008). Other approaches use lateral-overflow gate for the photodiode sense node (Decker et al., 1998) or lateral-overflow capacitor (Sugawa et al., 2005) for dual conversion gain readout. All these techniques enhance the DR to about 100 dB while having a reasonable drop of the signal-to-noise ratio (SNR) at the transition between high and low illumination. Logarithmic high dynamic range CMOS (HDRC) imagers exhibit up to 170 dB DR (Höfflinger, 2007; Burghartz et al., 2006) having a balanced but lower peak SNR over illumination. A considerably extended DR with linear response is achieved with light-to-frequency conversion (Yang, 1994) and self-reset pixels (Hirsch et al., 2017; Leñero-Bardallo et al., 2017; Peizerat and Arques, 2007; Kavusi et al., 2006; Wang et al., 2006). A DR of nearly 140 dB has been reported for a particular architecture of a self-reset pixel (Kavusi et al., 2006).

Our implementation of the self-reset pixel employing a pinned photodiode will be described in Sect. 2 where the maximum analyzable photo generated charge (Q_max) during the exposure time is extended to a multiple of the saturation charge of the photo diode (Q_sat) by asynchronous self-resets of the photo diode. This gives rise to a potentially higher peak SNR at high illumination levels compared to conventional CMOS imagers with 3T or 4T pixel architecture having an equal full well capacity (Q_sat). To increase the resolution, the remaining charge of the photo diode at the end of the exposure time is analyzed for improved low light sensitivity. We were able to achieve a DR of more than 120 dB using a pinned photodiode with low dark current and floating diffusion charge storage node.

The test chip which was designed and realized is explained in Sect. 3. Section 4 presents the measurement setup and the measurement results of the different single pixel cell variants and their opto-electronic characterization. In Sect. 5 the structure of the pixel cluster and its measurement results are shown.

2 Self-reset pixel

2.1 Pixel cell

Each self-reset pixel has an analog and a digital part and for all pixels one global control exists on-chip or off-chip (see Fig. 1). The analog part of the pixel cell is composed of a pinned photo diode, transfer and reset transistors, a floating diffusion (FD) node used as hold element, a comparator with high gain and a delay element with feedback path to reset the floating diffusion node. The advantages of the pinned photo diode are low leakage currents caused by a special pinned layer and an electronic shutter with a transfer gate to the floating diffusion node FD. For evaluation purposes the delay time within the feedback path is variable and can be adjusted from outside the chip by a bias voltage. The two stage comparator is quite important for the function of the pixel cell. It has an open loop gain of about 80 dB, is built up of 11 transistors and the input range is adapted to the voltage levels of the floating diffusion node. Its first stage is a telescopic transconductance amplifier. A tradeoff between the needed complexity and the available area for the layout had to be found. The comparator has a current consumption of about 300 nA. The layout of the analog part of the pixel cell needs an area of 20 × 20 µm².

Figure 1 Schematics of the analog and digital parts of one pixel cell and a global control for all pixel cells.

The digital part consists of several logic gates and a 10 bit counter which is designed as a linear feedback shift register with ten serially connected D-type flip-flops. The outputs of the seventh and tenth flip-flop are connected to a XNOR gate. In the counting mode the output of the XNOR gate is fed to the input of the first flip-flop whereas in the readout mode the input signal ser_in is connected to the input of the first flip-flop. This is needed to connect the counters of all the pixels of a pixel cluster (see Sect. 5) in series to do the readout of the data and to reset all the counters to zero or to preload to a specific value. The 10 bit counter value is shifted out serially on the signal ser_out. Due to the design of the counter as a linear feedback shift register the data needs to be decoded after readout.

2.2 Conversion concept using the self-reset pixel

The self-reset pixel with the analog and digital part was explained in the previous section. For image capturing it operates in four phases as depicted in Fig. 2. After an initial reset to SVDD the floating diffusion (FD) charge storing node is discharged during the first phase due to the photo current generated by the light exposed photodiode while the transfer gate is switched on. If the node reaches the reference voltage V_comp it will be reset to SVDD and the value of the counter in the digital part of this pixel is increased by one. This will continue until the end of the integration time of 10 ms. Thus after the end of phase 1 the value of the counter which contains the number of self-resets represents a coarse value of the measured light intensity. In phase 2 this counter value is read out and simultaneously the counter is reset to 0. At the end of phase 1 a residue charge remains on the floating diffusion node in the pixel, represented by the voltage V_FD on this node. This voltage is converted as a fine value of the pixel in phase 3 using a ramp analog to digital conversion. In phase 4 this fine value is read out from the counter and the counter is reset again. The combination of the 10 bit coarse and 10 bit fine values results in a linear 20 bit value.

Figure 2 Working principle of the self-reset pixel.

If the above described conversion procedure is done for a lot of different levels of illumination the coarse and the fine values can be illustrated in diagrams as a function of the illumination (see Fig. 3). It is clearly seen, that for a constant level of the coarse value, the fine values increase linearly with increasing illumination forming a saw tooth like curve with the amplitude ΔFV.

Figure 3 (a) Coarse and (b) fine value as a function of illumination.

3 Realized CMOS test chip

The test chip in Fig. 4 was fabricated in an advanced 0.18 µm CMOS technology for optical sensors with low leakage currents. The supply voltage of the analog parts is 3.3 V, the digital parts work with 1.8 V. The following elements have been integrated on the chip: 23 variants of single pixel cells (i), two different 16 × 16 pixel clusters (ii) and one digital to analog converter (iii). The clusters are described more in detail in Sect. 5. The 23 variants of single pixel cells are composed of different types of photo diodes (differences in layout), transistors (different threshold voltages), comparators (different topology), delay elements and counters/shift registers (static, dynamic, quasi static, static with latch). The reason of the different pixel variants will be addressed in more detail in the next paragraph. In the final implementation the digital to analog converter shall create the variable reference voltage (constant in phase 1, linear ramp in phase 3) at the node v_comp which is connected to the inverting input of the comparator in the self-reset pixels and used in the phases 1 and 3. To be more flexible during evaluation of the pixel cells in this test chip the reference voltage v_comp is not generated with the on chip DAC but by means of an external arbitrary waveform generator.

Figure 4 Realized CMOS test chip: (a) photograph of the packaged chip, (b) screenshot of the layout.

Pixel variants 18–23 implemented with a better high gain comparator than variants 1–17 have been selected for optical measurement to experimentally identify the best functional performance like DR, low light sensitivity, etc. Table 1 lists the pixel differences of the photodiode related transistors pixTG and sRST, the floating diffusion (FD) capacitance, the photodiode (PD) area and fill factor (in respect to the analog area of 20 × 20 µm²). Regular (RVT) or low (LVT) threshold voltage of transistor sRST (during phase 1) and pixTG (at the end of phase 1) give rise to a voltage shift on FD node through channel charge injection and clock feedthrough of the switched transistor gate in relation to the capacitance of the FD node (Razavi, 2001). Since the FD node voltage determines the transition point in time of the comparator output this is a sensitive implementation criterion. Also the leakage currents of the FD node itself, the connected transistors and the photodiode have impact on the FD node voltage during the phases 1–3 and hence on functional and optical performance. Other differences between the pixels concern the delay element and the layout pattern of analog pixel circuits.

Table 1 Realized self-reset pixel variants with high gain comparator using different photodiode related transistor types (RVT regular, LVT low threshold voltage) and design parameters of floating diffusion (FD) and photodiode (PD).

4 Measurement

4.1 Measurement setup

Figure 5 shows the principle of the measurement setup which was developed for the evaluation of the pixel cells variants shown in Fig. 4 part i. The time critical “analog” pixel signals pixRST, pixTG, pixFB and also the signal v_comp (including the analog ramp for phase 3) are externally driven by three programmable (arbitrary) waveform generators. Basically the pixel signals pixTG/RST/FB are digital signals with a high and low level only, but for the evaluation of the pixel cells the timing, the voltage levels and the rise and fall times are also important. Using the waveform generators here gives the required flexibility. The digital coarse and fine values need to be transferred during the phases 2 and 4 from the counter/shift register in the pixel cells to the PC for a computer based analysis of the data. This transfer is done with a mixed signal I/O (input/output) measurement box connected via USB interface with the PC. Especially the readout in phase 2 has to be done quite fast in less than 0.1 ms to avoid too much loss of charge from the floating diffusion node due to parasitic leakage current during this readout time. The variable illumination of the pixel cells is done by a controlled light source with intensity between 1 lx and 13 klx including 0 lx (light source off). The test chip was bonded into an 84 pin ceramic package and inserted in an evaluation board. On this board several voltages to be applied to the chip can be generated by voltage regulators or by digital to analog converters. Furthermore the chip signals can be easily accessed for measurement and stimulation. The voltage swing ΔV_FD of the node FD was set to 425 mV in phase 1 and the ramp of the signal V_comp in phase 3 had a swing of 1.7 V.

Figure 5 Measurement setup.

4.2 Opto-electronic characterization

The opto-electronic characterization of the pixel cells variant 18–23 is done in the following way: For each illumination level between 0 lx and 13 klx the measurement is repeated 16 times. The time lag between the measurements is about 12.5 ms. Thus for each illumination level a sequence of 16 coarse values (number of self-resets during phase 1) and 16 fine values (representing the remaining voltage of the node FD at the end of phase 1) is available. For the analysis of the data the average (or median) of the 16 coarse and 16 fine values is calculated. The standard deviation of the 16 fine values determines the temporal noise figure at each illumination level (note: exclusion of outliers at transitions between two adjacent coarse values and median filtering has to be applied to calculated noise curves).

Figure 6 Measured Coarse (a) and Fine Values (b) for 0–200 lx and the calculated OECF (c) with deviation from linearity (d) over the whole illumination range for the six different pixel cell variants 18–23.

Table 2 Measurement results of the opto-electronic characterization for the pixel variants with high gain comparator. Pixel variant 19 (bold font) is the favorite for being integrated in future work.

Figure 6a shows the measured (median) coarse values and Fig. 6b the fine values for low illuminations 0–200 lx of the total range 0–13 klx. Compared to Fig. 3 the principle behavior is as expected but with differences in CV-slopes, FV-offsets and ΔFV-amplitudes for the six pixel variants which are due to different pixel design parameters shown in Table 1. ΔFV-amplitudes are determined for each pixel individually (see Fig. 6b in comparison to Fig. 3) and are given in Table 2 together with the temporal dark noise SD0, noise SD1 at 100 lx, responsivity R and extracted sensitivity S. Additionally the input offset voltage of the comparators, which is expected within the range of ±100 mV, has impact on the CV-slopes and FV-offsets and will give rise to fixed-pattern noise (correctable static pixel-to-pixel variations) in an image sensor.

Combining the measured coarse and fine values for each illumination of the total range 0–13 klx results in a linear opto-electronic conversion function (OECF), given in Eq. (1):

$$\begin{array}{}\text{(1)}& \mathrm{OECF}\left(E\right)=\mathrm{CV}\left(E\right)\cdot \mathrm{\Delta }\mathrm{FV}+\mathrm{FV}\left(E\right),\end{array}$$

where CV(E) means the number of coarse values at the illumination E, FV(E) is the number of fine values at the illumination E and ΔFV is the amplitude of the saw tooth like curve of the fine value as introduced in Fig. 3.

Figure 6c shows the calculated OECF using Eq. (1) for each pixel variant with fine-tuned amplitude ΔFV from Table 2 over the measured illumination range 0–13 klx. Fine tuning of ΔFV was done such, that the calculated OECF curve is continuous at transition points at illuminations where CV has a positive step (by 1) while FV has a negative step (by −ΔFV) as one can recognize in Fig. 6a and b.

The deviation from linearity of the OECF can be calculated as given in Eq. (2):

$$\begin{array}{}\text{(2)}& \mathrm{rel}\mathrm{Error}\left(E\right)={\displaystyle \frac{\mathrm{OECF}\left(E\right)-\mathrm{RSL}\left(E\right)}{\mathrm{RSL}\left(E\right)}},\end{array}$$

where RSL(E) is the reference straight line connecting the start and ending point of the OECF at minimum and maximum illumination. With Eq. (2) Fig. 6d shows the rel. error with a deviation from linearity less than ±5 % for all pixel variants over the measured illumination range 0–13 klx.

It is interesting to notice that although the coarse value curves in Fig. 6a have different slopes, the calculated OECF curves in Fig. 6c, except for pixel variant 18, show nearly the same slope. This is due to the fact that the first derivative dOECF(E)∕dE of Eq. (1) is determined by the derivative of the fine values dFV(E)∕dE only since the derivative $\mathrm{dCV}\left(E\right)/\mathrm{d}E=\mathrm{0}$ when CV = const. This is true except at the transition points where CV has a positive step by 1 increasing the OECF value by ΔFV according to Eq. (1). Figure 6b confirms that all pixels, also pixel variant 18 for FV < 700, exhibit nearly equal FV-slope dFV(E)∕dE (e.g. visually compare the parallel curves at illumination 0–10 lx) which is determined (here in the context of the pixel variants) to a high extent on the pixel diode photocurrent and the floating diffusion capacitance as charge integrating node. This is represented by the ratio of photodiode area and FD-node capacitance. With values from Table 1 this results in 14.19 µm²/fF (pixel variants 18, 19, 22, 23) and 14.34 µm²/fF (pixel variants 20, 21) which is about 1 % difference only. The reason for the reduced slope of the OECF curve for pixel variant 18 comes from a reduced ΔFV value for that pixel due to non-linear distortion which can be seen in the green curve in Fig. 6b for FV > 700. Most probably the input stage of the differential comparator is beyond the limit of its linear input voltage range.

The measurement results of major characteristic values for the pixel variants are summarized in Table 2. The pixel variant 19 is the favorite for being integrated in future work. It has a very good sensitivity of 11 mlx @ SNR = 1 (Signal-to-Noise Ratio) and t_int=10 ms (integration time in phase 1) and a dynamic range of more than 121 dB (note: dynamic range was limited by the maximum possible illumination provided by the measurement setup).

Figure 7 Layout of the pixel cluster showing details of the pixel cell and the peripheral counter/shift register.

5 Pixel cluster

To be used in multi aperture imaging systems the pixel cluster in Fig. 7 was designed (two clusters are already shown in Fig. 4 part ii). For this cluster the analog and digital parts of the pixel cells were separated from each other. Only one signal line is needed to connect the two parts. In the middle of the cluster a matrix array of 16 × 16 analog pixel cell blocks was built. In the area above and below 256 corresponding static counters/shift registers were placed. These counters are daisy chain connected so that the counter output of the first pixel cell is connected to the input of the second one and so on. That means that the values of the 256 counters are read out on one serial signal line (SerOut256). Setting the input of the first counter (SerIn1) to 0 during the readout all counters will be reset to 0 at the end of the readout phase. The one-to-one connection between the analog and digital parts of the pixel cells and the daisy chain connection of the counters is done in that way that the pixels of the cluster are read out line by line from bottom to top and within each line pixel by pixel from left to right. The size of the complete cluster is 960 × 960 µm². The second cluster on the chip differs from the first one in such a way that it has an additional buffer between the analog and the digital part to drive the long interconnect line and to decouple the analog and the digital part to avoid potential crosstalk issues between long interconnecting lines of adjacent pixels. But the measurements have shown no crosstalk issues for both cases.

An optical lens with a focal length of 12.5 mm was mounted on top of one 16 × 16 pixel cluster to form a mini camera. A LED with a diameter of 3 mm in the distance of 40 cm from the lens tube was used as light source to be imaged by the cluster. With a reproduction scale on object side of 0.64 mm per pixel the size of the LED will be displayed approximately to 5 pixels. Due to the comparator with a lower gain in the pixel cell variant which was used in the pixel cluster only the coarse values could be generated. The serial sequence of 256 × 10 bit has been read out and software processed to be displayed in a 2-D image with a 16 × 16 resolution (see Fig. 8).

Figure 8 Image of a LED taken by a 16 × 16 pixel cluster.

6 Conclusion

Self-reset pixel cells with pinned photo diode for improved low light sensitivity in terms of 23 different variants of single pixel cells and two 16 × 16 pixel cell clusters have been realized on a test chip in a 0.18 µm CMOS image sensor technology. For the measurement setup an evaluation board with flexible external generation of the pixel signals, optical illumination and computer aided data logging and processing was used. Six variants of pixel cells with a high gain comparator have been characterized in detail. Also a pixel cluster with a serial readout scheme via shift registers was evaluated successfully.

The self-reset concept effectively extends the linear dynamic range to more than 121 dB in our favorite pixel variant which was limited in our case by the maximum possible illumination provided by the measurement setup. The maximum deviation from linearity is below 3.6 %. We achieved a very good low light sensitivity of 11 mlx, equivalent to 0.037 mW m⁻² (for monochromatic irradiance at 633 nm), at a frame rate of 80 frames per second (fps) due to the use of a pinned photo diode with very low leakage current. Its floating diffusion is hereby used as charge storage node. The photodiode size is 52.5 µm² while the non-optimized chip area per pixel of the pixel cluster consisting of the analog and digital part is equivalent to a 49 µm quadratic pixel pitch. This could be decreased to 34.4 µm still using static flip-flops as indicated in our future work below, resulting in an overall fill factor (FF) of 4.4 %.

In comparison to other self-reset pixels also fabricated in 0.18 µm CMOS technology Wang et al. (2006) achieved a linear dynamic range of 115 dB and a non-linear distorted one of 130 dB at much higher minimum irradiance level of about 30 and 1.8 mW m⁻², respectively. The pixel pitch is 23 µm with a photodiode area of 132 µm² (25 % FF), 2.5 times larger than ours, by employing smaller size dynamic flip-flops. To illustrate some other specification and trade-offs Kavusi et al. (2006) achieved a dynamic range of 138 dB with a larger pixel size of 30 µm × 150 µm for thermal imaging with a frame rate of 1000 fps. Leñero-Bardallo et al. (2017) achieved a dynamic range of 105 and 125 dB at 30 and 3 fps respectively with a pixel pitch of 25 µm and a FF of 10 %.

Our future work will implement the most promising pixel (pixel variant 19 in Table 2) to an array sensor with a resolution greater than VGA. Some work on this already shows that a pixel pitch of 34.4 µm for the combination of the analog and digital part (Fig. 7 left and right respectively) into a single rectangular block can be achieved. Here an advantage is that the conversion results are already stored as digital values in each pixel cell and can be read out serially. To ensure safe operation under all operating conditions, especially a wide temperature range, the counter will be built up of static flip-flops using standard cell layout. Therefore the counter consumes a larger part of the total pixel area. A further reduction of this area could be done by replacing the static standard cell D-type flip-flop counter by a dynamic counter with fewer transistors or by replacing the digital counter by analog concepts (Peizerat et al., 2009; Kleinfelder et al., 2001). On the other hand the division of the self-reset pixel in an analog and a digital part with only one signal in between them also predestines it for die-stacking technology (3D IC). The digital counter could be realized area efficiently using a CMOS technology node with smaller feature size and placed below the photosensitive analog sensor die (Suárez et al., 2010) thus achieving a smaller pixel size with increased fill factor of the photodiode of about 13 % (Table 1).