A modern phone or laptop camera sensor is made by building a grid of photodiodes on one silicon wafer and the readout and processing circuits on a second wafer, bonding the two face to face, thinning the pixel wafer so light enters from the back, then adding a color filter mosaic and a microlens over every pixel. The finished die is tested, mapped for defects, packaged, and assembled with a lens stack, an infrared cut filter, an autofocus actuator and a flex cable into a camera module.
This post walks the whole line from bare wafer to finished module, then follows the image into the firmware, drivers and permissions that decide who sees it. Our earlier post on how cameras evolved from CCD to stacked CMOS covers the history; this one stays on the factory floor.
Key takeaways
- A CMOS sensor turns light into electrons in a pinned photodiode, converts the charge to a voltage in each pixel, and digitizes it with ADCs usually placed at the bottom of each column.
- Back side illumination (BSI) flips and thins the pixel wafer so light avoids the wiring. One major sensor maker says its 2009 BSI product was roughly twice as sensitive as its front illuminated parts.
- Stacking builds pixels and logic on separate wafers. Hybrid copper to copper bonding (IEDM 2016) joins them through pads across the whole face; a 2017 ISSCC paper added a DRAM layer in between.
- The Bayer color filter mosaic (Kodak patent, published 1976) and the microlenses are made by photolithography on top of the finished pixel wafer.
- Every stage is tested: wafer probe, dark current and defect pixel mapping, then module calibration. EMVA 1288 is the standard method for measuring sensor performance.
- The image then passes over MIPI CSI-2 through sensor registers, an ISP, a kernel driver, a camera framework and an OS permission check. Every layer is software with its own supply chain.
- Indicators are only as trustworthy as what drives them: a 2014 USENIX paper disabled a laptop webcam LED by rewriting camera firmware from user space.
Timeline: the manufacturing milestones that matter
Each row is checked against a primary source in the Sources list.
| Year | Milestone | Why it matters for manufacturing | Source |
|---|---|---|---|
| 1969 | Willard Boyle and George Smith at Bell Laboratories invent the charge coupled device (CCD) | First practical way to read out a large grid of pixels; recognised with the 2009 Nobel Prize in Physics | Nobel Prize press release, 6 October 2009 |
| 1975 to 1976 | Bryce Bayer's "Color imaging array" patent filed (5 March 1975) and published (20 July 1976), assigned to Eastman Kodak | Defines the red, green, green, blue mosaic still deposited on most sensors | US 3,971,065 |
| 1997 | Eric Fossum's "CMOS image sensors: electronic camera-on-a-chip" in IEEE Transactions on Electron Devices, vol. 44, no. 10, pp. 1689 to 1698 | Makes the case for active pixel sensors built in standard CMOS processes, with readout and control on the same chip | IEEE paper metadata (Crossref) |
| 2009 | A large image sensor maker ships a back illuminated CMOS sensor, roughly twice the sensitivity of its front illuminated parts | Adds wafer bonding and back side thinning to the standard process flow | Sensor maker technology page |
| 2016 | "Novel stacked CMOS image sensor with advanced Cu2Cu hybrid bonding" presented at IEDM (Kagawa, Fujii, Aoyagi, Kobayashi and others) | Replaces through silicon vias at the edge of the die with dense copper pads across the bonded face | IEDM 2016 paper metadata |
| 2017 | "A 1/2.3inch 20Mpixel 3-layer stacked CMOS Image Sensor with DRAM" presented at ISSCC (Haruta and others) | Adds a memory layer so frames can be read off the pixels fast and buffered on the sensor | ISSCC 2017 paper metadata |
| June 2021 | EMVA 1288 Release 4.0 takes effect | Current version of the standard method for characterizing sensors and cameras | EMVA |
| 15 December 2025 | MIPI CSI-2 v4.2 adopted | Current camera serial interface specification; adds event based vision sensor support | MIPI Alliance |
How a CMOS image sensor turns light into a number
Before the factory, the physics, because every manufacturing step exists to protect one part of this chain.
The photodiode
Silicon absorbs visible light and, for each photon absorbed in the right place, frees an electron. A photodiode is a reverse biased junction that sweeps those electrons into a small well and holds them for the length of the exposure. Stanford's EE 392B lecture notes on silicon photodetectors define the two numbers that matter most here. Quantum efficiency is the fraction of incoming photon flux that ends up as photocurrent, and it depends on wavelength because blue light is absorbed close to the surface while red and near infrared light travel deeper before being absorbed. Dark current is the current the photodiode produces with no light at all, from thermal generation and defects, and the same notes put it bluntly: dark current is bad, because it adds noise and fills the well with charge that has nothing to do with the scene.
That is why so much of the fab process is about the surface of the silicon. Defects at the interface between silicon and oxide generate dark current. Damage from etching and implantation generates dark current. Metal contamination generates hot pixels. A sensor fab is a CMOS logic fab with an obsession about leakage.
The pinned photodiode and the four transistor pixel
Most sensors today use a pinned photodiode: a buried n type region covered by a thin, heavily doped p type layer at the surface. The p+ layer keeps the charge away from the defect rich surface, which cuts dark current, and it lets the photodiode be emptied completely when a transfer gate opens. The EE 392B notes on CMOS image sensors walk through the pinned diode active pixel: before a row is read, the floating diffusion node is reset; then the transfer gate turns on and moves the accumulated charge onto that node, where it changes the voltage.
The common pixel has four transistors:
- Transfer gate moves charge from the photodiode to the floating diffusion.
- Reset transistor sets the floating diffusion to a known voltage.
- Source follower buffers the floating diffusion voltage so it can drive a long column wire.
- Row select connects that buffered voltage to the column line when the row is being read.
Many designs share the reset, source follower and select transistors between two or four photodiodes to leave more area for light collection. Reading the reset level first and the signal level second, then subtracting them, is called correlated double sampling. It removes the reset noise and the fixed offset of each pixel's source follower.
The analog to digital converter
The voltage on each column line has to become a number. In a column parallel design, every column (or every few columns) has its own ADC at the bottom of the array, so a whole row is converted at once. The EE 392B notes list this ability to integrate analog and digital circuits on the same chip as the main advantage CMOS has over CCDs: fewer chips in the camera, lower power, faster readout and more programmability. A sensor maker's technology pages list column parallel A/D conversion as one of the common technologies under all of its image sensors.
Rolling shutter and global shutter
With column parallel ADCs, rows are read one after another. The exposure of each row starts and ends a little later than the row above it. That is a rolling shutter, and it is why fast moving objects lean or wobble in phone video.
Worked example: rolling shutter skew. Take a 12 megapixel sensor with 4,000 columns and 3,000 rows. If the readout takes 10 microseconds per row, the last row starts its exposure 3,000 times 10 microseconds, or 30 milliseconds, after the first. A car crossing the frame at 20 metres per second moves 0.6 metres in that time, so its vertical edges will slant visibly. Halve the row time to 5 microseconds and the skew falls to 15 milliseconds. This is one reason stacking matters: putting more ADCs and faster logic underneath the pixels shortens row time.
A global shutter exposes every pixel at the same moment and stores the charge (or a voltage) in a memory element inside each pixel until it can be read. The cost is area and design effort: the storage node must be shielded from light and must not leak while the rows ahead of it are being read. Global shutter sensors are common in machine vision and in some automotive and face recognition cameras.
From number to image
What leaves the sensor is raw: one value per pixel, each seen through one color filter. The image signal processor (ISP) subtracts the black level, corrects lens shading, hides defective pixels, demosaics, balances white, reduces noise and tone maps, using numbers measured during manufacturing.
How the pixel array is built on the wafer
An image sensor fab starts with the same ingredients as any other chip fab: a polished single crystal silicon wafer, usually with an epitaxial layer grown on top, and a long sequence of lithography, implant, deposition and etch steps. Our post on how semiconductors are made from sand to chip covers the wafer, lithography and etch basics. Here are the steps that are specific to image sensors.
Isolation between pixels
Each pixel must keep its electrons and its light. Shallow trench isolation separates transistors as in any CMOS process. Small back illuminated pixels add deep trench isolation: narrow trenches etched deep into the silicon around each photodiode and filled with oxide, sometimes with metal, to stop light and charge crossing into the neighbour (crosstalk). Etching damages the trench walls, so they are passivated to keep dark current down.
Implants that form the photodiode
Ion implantation forms the photodiode: a deep n type implant for the charge well and a shallow p+ implant to pin the surface. Energies and doses set the full well capacity and how completely the transfer gate can empty the pixel. Small errors show up later as image lag or as a bright pixel in every dark frame.
Transistors and metal layers
Next come the pixel transistors and, in a non stacked design, the readout chain around the array: row drivers, column ADCs, timing and the interface. Copper or aluminium interconnect layers go on top. In a front illuminated sensor the metal must leave an open path over each pixel for light.
Front side illumination and back side illumination
| Aspect | Front side illumination (FSI) | Back side illumination (BSI) |
|---|---|---|
| Light path | Through microlens, color filter, then the metal and dielectric stack, then into silicon | Through microlens and color filter straight into the thinned silicon from the back |
| Wafer thickness at the end | Full thickness wafer | Pixel silicon thinned to a few micrometres, supported by a bonded wafer |
| Extra process steps | None beyond standard CMOS plus color filters and microlenses | Wafer to wafer bonding, back grinding, chemical mechanical polish or etch back, back side passivation, back side anti reflection coating, pad openings through the thinned silicon |
| Sensitivity at small pixel sizes | Falls as the metal openings shrink relative to the pixel | Higher, because wiring no longer blocks light |
| Typical weak points | Light blocked or scattered by the wiring; poor response at steep angles | Back surface defects raise dark current; color crosstalk without good isolation and alignment |
| Where you find it | Larger pixel sensors, some industrial and low cost parts | Most small pixel sensors, including phone cameras |
One sensor maker's own technology page describes the trade honestly. Its 2009 back illuminated sensor reached roughly double the sensitivity of its front illuminated products because metal wiring and transistors were no longer in the light path. The same page says that back illuminated structures generally bring problems from the device structure and manufacturing process (noise, dark current, defective pixels and color mixture) and that it needed a new photodiode structure and high precision alignment to overcome them.
The BSI process flow, step by step
- Build the front side. Photodiodes, transistors and metal are made as in an FSI flow, with no need to leave light openings in the metal.
- Bond a carrier. Flip the pixel wafer face down onto a support wafer, or onto the logic wafer in a stacked design.
- Thin the pixel wafer. Grind, then polish or etch, until only a few micrometres of silicon remain. Thickness uniformity matters because it sets how much red and near infrared light is absorbed.
- Passivate the back surface. The freshly exposed silicon generates dark current, so a passivation layer, often a high k dielectric with fixed negative charge, does for the back what the p+ pinning layer does for the front.
- Add coatings and light shields. Anti reflection films cut reflection; a metal grid blocks steep light; fully covered pixels become optical black references for the black level.
- Open the pads through the thinned silicon to the front side metal.
- Build the color filters and microlenses on the back, aligned to the photodiodes below.
How wafer stacking works
A plain BSI sensor still has its readout circuits around the edge of the pixel array on the same silicon. A stacked sensor goes further: the support wafer is replaced with a working logic wafer.
Two layers: pixel wafer on logic wafer
A sensor maker's page on stacked structure explains the idea in two sentences. Instead of a blank supporting substrate under the back illuminated pixel section, a chip carrying the signal processing circuit is used, and stacking it under the pixels makes it possible to fit a large circuit on a small chip. Because each section is made on its own wafer, each can use the process that suits it: a pixel process tuned for low dark current and good photodiodes, and a logic process tuned for fast, dense transistors.
That separation is the real manufacturing win. Pixel processes want thick oxides, special implants and gentle thermal budgets. Logic processes want thin gates and many metal layers. Doing both on one wafer forces compromises. Two wafers remove most of them. The pixel wafer keeps the photodiodes, transfer gates and often the in-pixel transistors. The logic wafer carries the ADCs, row drivers, timing generator, digital processing, the serial interface, and the control registers that firmware writes to.
Joining the wafers: through silicon vias versus hybrid bonding
The two wafers must be connected electrically, millions of times over in some designs. There are two main approaches.
Through silicon vias (TSV). After bonding and thinning, holes are etched through the pixel silicon down to the metal on the logic wafer and filled with metal. They are reliable but take up dedicated area, usually in a ring around the array, which limits how many connections you can make and where.
Hybrid bonding (Cu to Cu). Each wafer's top surface is finished with copper pads set into a very flat dielectric. The two faces are aligned, pressed together so the dielectric surfaces bond, then annealed so the copper pads expand and fuse into one metal connection. The IEDM 2016 paper "Novel stacked CMOS image sensor with advanced Cu2Cu hybrid bonding" is a widely referenced early publication on the technique. A sensor maker's page on Cu-Cu connection puts the benefit in manufacturing terms: copper terminals on the stacked faces connect pixel chip and logic chip directly, so no connecting parts have to pass through the pixel chip and no special connection area is needed, which allows smaller sensors, higher productivity and more freedom in where the terminals go.
| Interconnect | Where connections sit | Density | Process steps added | Main manufacturing risks |
|---|---|---|---|---|
| Through silicon via | Dedicated area, typically around the array edge | Lower; limited by via size and keep out zones | Deep via etch through thinned silicon, liner, barrier, metal fill, polish | Via voids, stress cracking, area cost |
| Hybrid Cu to Cu bond | Anywhere across the bonded faces, including under the array | Much higher; pads on a fine pitch | Surface planarization to near atomic flatness, plasma activation, precise wafer to wafer alignment, anneal | Particles causing voids, misalignment, copper dishing or protrusion, bond strength |
The flatness requirement is the hard part. A single particle between two wafers leaves an unbonded bubble that can wipe out dozens of dies. Hybrid bonding lines therefore run cleaner than most back end steps, with surface inspection immediately before the bond.
Three layers: adding DRAM
The ISSCC 2017 paper "A 1/2.3inch 20Mpixel 3-layer stacked CMOS Image Sensor with DRAM" (Haruta and coauthors) described pixel, DRAM and logic layers in one sensor. With memory on the sensor, the array can be read very fast into DRAM, shortening rolling shutter skew and allowing high frame rate bursts, while the slower link to the processor drains the buffer afterwards. The cost is a third wafer from a different kind of fab and another bonding and thinning sequence.
How the color filter array and microlenses are made
Silicon cannot tell colors apart. It only counts photons. Color comes from a mosaic of tiny filters, one per pixel, sitting directly on the sensor.
The Bayer pattern and its variants
Bryce Bayer's patent, "Color imaging array", filed by Eastman Kodak on 5 March 1975 and published on 20 July 1976, describes the arrangement still used in most cameras: a repeating two by two block with two green filters, one red and one blue. Green gets twice the share because human vision takes most of its sense of detail from the green middle of the spectrum.
Worked example: what a Bayer sensor actually samples. A 12 megapixel Bayer sensor has 6 million green pixels, 3 million red and 3 million blue. The full color 12 megapixel image is reconstructed by demosaicing: for every pixel, the ISP estimates the two missing colors from neighbours. Detail at the finest scale in red or blue is therefore interpolated, which is why fine red text or a blue patterned shirt can show color fringes.
Variants change the grouping rather than the idea:
- Quad Bayer (2 by 2 grouping). Each color filter covers a block of four pixels. In low light the four are binned into one larger effective pixel. In good light the ISP remosaics them to a standard Bayer layout for full resolution. One sensor maker lists "Quad Bayer Coding" among its mobile sensor technologies.
- RGBW and other patterns. Some designs add clear (white) pixels for sensitivity, at the cost of more complex color processing.
- Phase detection pixels. Some pixels are partly masked, or share one microlens across two photodiodes, so the sensor can measure focus directly. These pixels have to be known to the ISP so it can correct them in the final image.
Depositing the filters
The filters are pigmented photoresists, one material per color, patterned by photolithography once per color: planarize, spin on the green resist (it covers half the array), expose and develop, then repeat for red and blue, each aligned to the pixels and to the previous colors, and finish with a second planarization layer. A filter shifted by a fraction of a pixel puts the wrong color over part of the neighbouring photodiode, which calibration can only partly undo, so some designs add a thin oxide or metal grid between filters.
Building the microlenses
A microlens over each pixel focuses light onto the photodiode. The usual method is to pattern a transparent resist into one island per pixel, heat it so each island reflows into a dome, then either keep it or etch the dome shape into a harder layer below, and add an anti reflection coating.
Lenses near the edge of the array are shifted toward the centre, because light from the camera lens reaches edge pixels at a steeper angle. This chief ray angle matching must agree with the module lens, so sensor and lens are designed as a pair.
The color filter and microlens layers are organic materials on top of an otherwise finished chip. They limit later process temperatures, scratch easily, and are why sensor dies cannot go through some hot assembly steps other chips tolerate.
How the sensor is tested and characterized
A sensor that powers on is not a good sensor. Every die gets measured.
Wafer level probe
Before the wafer is cut, a probe card touches the pads of each die and runs electrical and optical tests: supply currents, register read and write over the control interface, a dark frame and one or more uniformly lit frames. Dies that fail are marked so they are not packaged.
Dark current and defect pixel mapping
With the light blocked, the tester captures frames at several exposure times and sometimes several temperatures. Mean dark signal growth with exposure gives dark current. Pixels well above the mean are hot pixels. Pixels that do not respond to light are dead pixels. Pixels that saturate at a lower level or behave nonlinearly are flagged too.
A die can have a small number of defective pixels and still be sold. Their coordinates are written into a defect map, stored either in one time programmable memory on the sensor itself or in a memory chip in the module, so the ISP can replace each one with an estimate from its neighbours. The map is part of the device's identity. If it is lost or corrupted, the same sensor suddenly shows bright speckles.
EMVA 1288: a common language for measurements
The European Machine Vision Association publishes EMVA 1288, which it calls the standard for measurement and presentation of specifications for machine vision sensors and cameras. Its aim is a unified method to measure, compute and present parameters so buyers can compare cameras fairly. Release 4.0 took effect in June 2021. The standard has also been adopted at ISO level as ISO 24942, according to EMVA's standards pages.
The core measurements are the ones you would expect from the physics above: quantum efficiency, temporal dark noise, system gain (digital numbers per electron), saturation capacity, dynamic range, signal to noise ratio, dark current and its doubling temperature, and spatial nonuniformity (dark signal nonuniformity and photo response nonuniformity). Phone camera makers do not usually publish EMVA 1288 sheets, but the same kinds of measurement drive their production tests.
Worked example: reading a photon transfer curve. Suppose a test bench measures, at increasing light levels, a mean signal and a temporal variance (both in digital numbers after subtracting the dark level). At a mean of 400 DN the variance is 100 DN squared; at a mean of 800 DN the variance is 200 DN squared. In the shot noise limited region the variance grows linearly with mean signal, and the slope is the system gain K, here 100 divided by 400, or 0.25 DN per electron. So a mean of 800 DN corresponds to 800 divided by 0.25, or 3,200 electrons. If the signal clips at 3,000 DN, the saturation capacity is about 12,000 electrons. If the dark temporal noise is 2 DN, that is 8 electrons, and the dynamic range is about 12,000 divided by 8, or 1,500 to 1, which is about 63.5 dB (20 times log10 of 1,500). The numbers are illustrative, but this is exactly the arithmetic EMVA 1288 formalizes.
How the sensor becomes a camera module
Our post on how a smartphone is made covers final phone assembly. The camera module is its own sub assembly, usually built by a dedicated module maker in a clean room.
Packaging the die
Tested wafers reach a package by one of a few routes. In chip on board, the bare die is attached and wire bonded inside the module, which gives the lowest height but leaves the die exposed until the lens holder is sealed. In chip scale packaging, a glass cover is bonded over the sensor at wafer level and connections are brought to the back, often through silicon vias, before dicing. In flip chip, the die is mounted face down on a substrate with an opening for light. Whichever route is used, a particle that lands on the microlens array during dicing or die attach becomes a permanent dark spot in every photo.
The lens stack and IR cut filter
A phone lens is a barrel of several molded plastic or glass elements, sometimes with a periscope prism for telephoto. Between lens and sensor sits an infrared cut filter, a glass plate with an interference coating. Silicon responds well into the near infrared, so without it skin and foliage render in the wrong colors. Night vision and face recognition cameras omit it or use a narrow pass band.
Autofocus and optical image stabilization
Most phone modules focus with a voice coil motor that pushes the lens barrel along its axis against springs. Optical image stabilization (OIS) shifts the lens or the whole sensor sideways to cancel hand shake, using a gyroscope signal. Each actuator has its own driver chip on the control bus, with its own firmware or calibration tables.
Active alignment
Rather than trust mechanical tolerances, module lines power the sensor, image a test chart through the lens, compute sharpness across the field, and move the lens in six axes until focus and tilt are best before ultraviolet curing adhesive locks it in place. It is one of the slowest and most important steps on the line.
Flex cable and connector
A flexible printed circuit carries power, clock, the control bus and the MIPI data lanes to the main board, and often a small EEPROM holding the module's calibration data and identity.
Module level calibration
Each finished module is calibrated, and the results go into its EEPROM or the sensor's own memory: lens shading (corner falloff and color shift, measured on a flat field), per module color gains from known light sources, autofocus positions for infinity and close focus, OIS gyroscope and actuator gains, and a check of the defect and phase detection pixel tables. On phones with several cameras, the relative position of each module is measured after phone assembly so depth and zoom transitions line up.
Step table: from wafer to finished camera module
| Stage | What happens | Key test or calibration | What a failure looks like later |
|---|---|---|---|
| 1. Starting wafer | Polished silicon wafer with an epitaxial layer, inspected for defects and contamination | Particle and metal contamination checks | Hot pixels, clusters of white spots |
| 2. Pixel front end | Isolation (shallow and deep trench), photodiode implants, pinning layer, transfer gates, pixel transistors | In line parametric tests on test structures | High dark current, image lag, low full well |
| 3. Pixel back end of line | Metal interconnect and dielectric layers; hybrid bond pads on the top surface for stacked designs | Electrical test structures, surface flatness | Shorts, open rows or columns |
| 4. Logic wafer (stacked) | ADCs, timing, digital processing, interface and registers built in a logic process; bond pads on top | Logic wafer probe | Missing columns, interface errors |
| 5. Optional DRAM wafer | Memory layer for three layer stacks | Memory test | Frame corruption at high speed |
| 6. Wafer bonding | Align and bond pixel wafer face down to logic wafer; anneal for Cu to Cu bonds | Bond void inspection (acoustic or infrared), alignment metrology | Dead regions, intermittent connections |
| 7. Thinning | Grind and polish pixel wafer to a few micrometres | Thickness uniformity mapping | Color shifts and red response variation across the frame |
| 8. Back side passivation and coatings | Passivation, anti reflection coating, light shield grid, optical black pixels | Dark current check on monitor dies | Dark current, flare |
| 9. Pad open and TSV (if used) | Etch to bond pads; vias for chip scale packages | Electrical continuity | Open connections |
| 10. Color filter array | Planarize; deposit, expose and develop each color resist; top planarization | Overlay alignment metrology, color uniformity | Color crosstalk, color shading |
| 11. Microlenses | Pattern lens resist, reflow or etch, anti reflection coating; chief ray angle shift toward edges | Lens profile and shift measurement | Corner shading, low sensitivity |
| 12. Wafer probe | Power, register access, dark and lit frames per die | Dark current, defect pixel count, gain, noise | Rejected dies |
| 13. Defect map | Write hot, dead and phase detection pixel coordinates to on-chip one time programmable memory | Readback check | Bright speckles if lost |
| 14. Dicing and packaging | Saw or laser dicing; chip on board, chip scale package or flip chip | Visual and particle inspection | Dust spots on every image |
| 15. Module assembly | Attach lens holder, IR cut filter, voice coil motor, OIS actuator, flex | Mechanical and electrical checks | Focus or OIS faults |
| 16. Active alignment | Six axis lens alignment against a live test chart, UV cure | Sharpness across the field | Soft corners, tilt |
| 17. Module calibration | Lens shading, color, autofocus, OIS; data stored in EEPROM or sensor memory | Pass and fail against targets | Color mismatch between units |
| 18. System calibration | After phone or laptop assembly: multi camera alignment, ISP tuning check | Final image quality tests | Bad depth maps, zoom jumps |
Where firmware and software sit
From here on, everything is software, and that is where most real world camera bugs and privacy failures live.
Sensor registers
A sensor is configured through hundreds or thousands of registers written over I2C or the newer I3C: clock tree, frame size, crop, binning, exposure, analog and digital gain, test patterns and output format. Many sensors also need a long initialization table from the maker to set internal analog bias points, which on Linux often ends up hard coded in the kernel driver.
The Linux kernel documentation for camera sensor drivers is clear on one rule. The sensor's internal PLL and dividers are configured from two hardware specific inputs, the external clock frequency and the link frequency, normally supplied by system firmware, and no other frequencies should be used because those clocks leave the system on chip and a wrong one can cause harmful effects elsewhere. Sensor drivers must also use runtime power management, because the sensor shares a pipeline with devices that power up and down together.
The link: MIPI CSI-2
Pixel data travels from the sensor to the processor over MIPI CSI-2, the camera serial interface. According to the MIPI Alliance, CSI-2 is lane scalable and usually runs on the C-PHY or D-PHY physical layers for short links inside a device, but can also run over the A-PHY long reach serializer link, up to 15 metres, for cars and industrial systems. It supports up to 32 virtual channels so one link can carry several streams or data types, which is how multi exposure HDR and sensor fusion data share a cable. The current version is CSI-2 v4.2, adopted on 15 December 2025, and all versions are backward compatible. MIPI also lists a Camera Command Set, a Camera Service Extensions specification and a Camera Security specification and profiles alongside CSI-2, which matters for automotive cameras where image authenticity is a safety question.
The image signal processor
The ISP is usually a hardware block in the system on chip, though some sensors and USB webcams carry their own. Fixed hardware does the pixel math; firmware or a host library runs the "3A" loop (auto exposure, white balance and focus) that sets the next frame's registers and parameters. Its behaviour is mostly data: noise curves, color matrices, lens shading tables, sharpening and tone curves, produced by image quality engineers for each sensor and lens pair and shipped as tuning files inside the firmware or vendor partition. Two phones with the same sensor can look completely different because of tuning alone.
Kernel drivers and media frameworks
On Linux, the sensor is a V4L2 sub device. The kernel's V4L2 sub device documentation describes these as the building blocks for hardware that is really a chain of separate chips and blocks, each with pads and formats, connected into a media graph. A typical graph runs from the sensor sub device, through a CSI-2 receiver, into the ISP and out to video capture nodes that applications can open.
Configuring that graph by hand is hard, which is why libcamera exists. Its project page describes it as an open source camera stack and framework for Linux, Android and ChromeOS, and explains the history: control of camera processing is based on algorithms that must run on a programmable processor, traditionally a dedicated microcontroller in the camera, but in embedded devices those algorithms moved to the main CPU to save cost. libcamera carries the pipeline handlers and the open image processing algorithms that make the 3A loop work on the host.
On Android, the camera service talks to a vendor camera HAL. The Android Open Source Project documentation for camera covers the HAL architecture, the 3A modes and state transitions, metadata and controls, and stream configuration. The vendor HAL, ISP firmware and tuning files are usually closed source. Laptop webcams are often USB Video Class devices with their own sensor, small ISP and microcontroller firmware, which matters for privacy, as the next section shows.
How camera indicators, kernel frameworks and OS permissions protect privacy, and where they have failed
There are several layers of protection between a camera and an app that wants it.
1. The operating system permission
Mobile operating systems treat the camera as a runtime permission. Android requires the CAMERA permission and, according to the Android 12 behavior changes page, supported devices running Android 12 or higher let users turn camera access off for all apps with a single toggle in Quick Settings or the Privacy settings page. The Android developer guidance on explaining access to sensitive information covers how apps should behave when the toggle is off.
2. The software indicator
Android 12 also introduced status bar indicators: when an app uses the camera or microphone, an icon appears. The Android Open Source Project page on privacy indicators adds the detail that in Android 12 or higher, camera and microphone use counts as active if the system marks it as running or if it is less than five seconds old. The icon then turns into a dot, and tapping it shows which app is using the camera.
3. Hardware and firmware backed indicators
A software indicator is only as trustworthy as the operating system drawing it. Laptops traditionally use an LED wired near the camera. Apple's Platform Security Guide documents a hardware microphone disconnect on Mac laptops and on iPad models from 2020 onward, and notes that the camera is not disconnected in hardware when the lid is closed because its view is completely blocked. The same guide has a separate section on the Mac on-screen camera indicator light.
Where indicators have failed
The best documented failure is academic. In "iSeeYou: Disabling the MacBook Webcam Indicator LED" (USENIX Security 2014), Matthew Brocker and Stephen Checkoway of Johns Hopkins University showed that on a class of internal webcams used in some older MacBook and iMac models, the indicator LED could be turned off so video was captured with no visual cue. The method was reprogramming the firmware that ran on the camera's own microcontroller, and it could be done from user space by an unprivileged application. The same firmware reprogramming allowed a virtual machine escape: malware in a guest made the camera present itself as a USB keyboard to the host. The lesson for manufacturing is direct. If the LED is driven by the same firmware that an attacker can rewrite, it is a software indicator with a light bulb attached. Indicators should be tied to camera power in hardware, and camera firmware updates should be signed and verified by the device.
Software indicators have their own limits. They can only show what the operating system's camera service knows about. Drivers or privileged system components that open the camera device directly, below the framework, may not trigger them, which is one reason platform makers restrict which processes can reach camera device nodes.
| Protection layer | What it guards against | What it cannot guard against |
|---|---|---|
| OS runtime permission | Ordinary apps using the camera without consent | Malware with system level privileges; compromised drivers |
| Global camera toggle | Every app at once, including ones with permission | Code below the camera framework |
| Software status indicator | Silent use by permitted apps | A compromised OS that does not draw it |
| LED driven by camera firmware | Casual use | Firmware rewritten by an attacker (as shown in 2014) |
| LED or indicator tied to sensor power in hardware | Use while the user thinks the camera is off | An attacker who can watch while the user knows it is on |
| Physical shutter or power switch | Everything while closed | Use while open |
Step by step: inspect your own camera pipeline
These commands are read only. They list devices and settings and do not change firmware or erase anything. Run them on a Linux machine you own.
1. List video devices and the driver behind each one.
# Install the V4L2 utilities (Debian or Ubuntu)
sudo apt install v4l-utils
# Which video and media devices exist, and which driver owns them
v4l2-ctl --list-devices
# Formats a capture node offers
v4l2-ctl -d /dev/video0 --list-formats-ext
2. See the media graph (sensor, CSI-2 receiver, ISP). On systems with a CSI-2 sensor (single board computers, many newer laptops), the media controller shows the chain of sub devices described above.
# Print the full media graph: entities, pads, links and formats
media-ctl -d /dev/media0 -p
3. Read the sensor's controls. Exposure, gain, test patterns and link frequency are exposed as V4L2 controls on the sensor sub device.
v4l2-ctl -d /dev/v4l-subdev0 --list-ctrls-menus
4. Ask libcamera what it sees. If your distribution ships libcamera, its cam utility lists cameras and their properties.
cam --list
cam --camera 1 --list-properties
5. Check which process has the camera open. This is the Linux equivalent of a privacy indicator, and it reads the kernel's view directly.
# Who holds the capture device right now
sudo fuser -v /dev/video*
# or
sudo lsof /dev/video0
6. Find the camera firmware and driver versions you are running. For USB webcams, lsusb shows the vendor, product and device release (often the firmware version). The kernel log shows which driver bound the device.
lsusb -v 2>/dev/null | grep -E "idVendor|idProduct|bcdDevice|iProduct"
sudo dmesg | grep -iE "uvc|camera|csi"
7. On Android, read the camera service state. With developer options and USB debugging enabled on your own device:
adb shell dumpsys media.camera | head -100
The output lists camera devices, the HAL version and recent client connections.
8. Measure dark current yourself (optional). Cover the lens fully, set a fixed long exposure and gain with v4l2-ctl, capture a few raw frames, and compute the mean and standard deviation per frame. Repeat at two exposure times. The increase in mean signal divided by the increase in exposure time is your dark signal rate, and pixels many standard deviations above the mean are your hot pixels. That is a small scale version of the wafer probe test in the step table.
Common mistakes
- Treating the sensor as the whole camera. Image quality comes from sensor, lens, alignment, calibration data and ISP tuning together.
- Comparing megapixels instead of pixel performance. Quantum efficiency, read noise, full well and dark current decide low light results; EMVA 1288 exists because headline numbers were not comparable.
- Assuming a camera LED is hardware. Ask whether it is wired to sensor power or driven by firmware. The 2014 iSeeYou work shows why.
- Losing the calibration data. Swapping a module without its defect map and shading data, or flashing the wrong tuning, produces speckles and color casts that look like hardware faults.
- Running a sensor at an unplanned clock. The kernel documentation says only the pre determined frequencies should be used.
- Leaving camera firmware out of inventory. USB webcams and modules run versioned code from a vendor, with an update path. Track it like a driver.
- Trusting a software toggle to stop privileged code. It stops apps, not a compromised driver or system service.
FAQ
How are camera sensors made?
In a semiconductor fab. Photodiodes and pixel transistors go on one wafer and readout logic on another; the wafers are bonded, the pixel wafer is thinned so light enters from the back, and color filters and microlenses are added. Tested dies are packaged and assembled with a lens, IR filter and focus motor into a module.
What is the difference between a CCD and a CMOS image sensor?
A CCD shifts charge across the chip to one output. A CMOS sensor converts charge to voltage in each pixel and digitizes it on the same chip, in a standard CMOS process, which made cheap, low power cameras possible. The CCD was invented in 1969 and won the 2009 Nobel Prize in Physics.
What does back side illuminated (BSI) mean?
Light enters the silicon from the side opposite the wiring, after the pixel wafer is flipped, bonded and thinned to a few micrometres, so metal no longer blocks it.
What is a stacked CMOS image sensor?
A sensor whose pixel array and processing circuits sit on separate bonded wafers instead of side by side. Some add a DRAM layer for fast capture, as described at ISSCC in 2017.
What is hybrid bonding in image sensors?
Joining two wafers by bonding their flat dielectric faces and fusing copper pads set into them, giving dense copper to copper connections anywhere across the die without through silicon via areas.
Why do camera sensors use a Bayer filter?
Silicon counts photons but cannot see color. A red, green and blue mosaic with twice as many green filters lets one sensor capture color, and the ISP fills in the missing colors at each pixel by demosaicing.
What are hot pixels and dead pixels?
Hot pixels show signal in the dark from high local dark current; dead pixels do not respond to light. Both are mapped during testing so the ISP can hide them, and new hot pixels can appear with age and heat.
Can a webcam be on without the light?
If the light is tied to camera power in hardware, it should not be. If camera firmware drives it, researchers showed in 2014 that rewriting that firmware could switch the light off on some older laptop webcams. A physical shutter removes the question.
What is an ISP tuning file?
The data that tells an image signal processor how to handle one sensor and lens pair: noise reduction, color matrices, lens shading, sharpening and tone curves, shipped with the device firmware.
What interface connects a camera sensor to the processor?
Usually MIPI CSI-2 over the C-PHY or D-PHY physical layer, with I2C or I3C for register control. Cars can carry CSI-2 over the A-PHY link up to 15 metres, according to MIPI.
Where Safeguard fits
Safeguard does not make, test or analyze image sensors, and it does not scan camera firmware or hardware. The fit is with the software around the camera. The ISP libraries, camera HALs, libcamera based pipelines, media frameworks, image codecs and the apps that consume camera frames are software with open source dependencies, and the same supply chain hygiene that applies to any application applies to them. If your team builds a camera app, a video pipeline, an embedded Linux image for a device with a camera, or a container that processes images, Safeguard can scan that source and those images for known vulnerable components, show which projects carry a given version, and open fix pull requests when your team turns remediation on. Our post on how OLED and LCD displays are made covers the other end of the imaging chain if you want the matching factory tour.
A camera pipeline is mostly software: image codecs, media frameworks, ISP host libraries and the apps on top all pull open source components, and a rebuilt device image pulls them all again. The sensor silicon is outside Safeguard's scope, but the dependencies in the code you build around it are not.
Fix it without waiting for upstream
Whatever the vulnerability, most of the risk sits in two gaps. The first is the time between a fix being published and that fix actually running in your production builds. The second, and worse, is the time before any fix exists at all, while a maintainer works out a patch and a release schedule. Waiting through both gaps is a habit, not a requirement. Safeguard gives teams three ways to close them.
1. Autonomous remediation: pull requests, not tickets
When a fixed version exists, Safeguard's Griffin models open a tested, policy gated pull request that moves the affected dependency to the safe version. You can run it on one repository or in bulk across every project that carries the vulnerable version. Remediation covers eight categories, including dependencies, licenses, security posture and findings. Reachability analysis ranks the vulnerable code paths your application actually calls first, so the first pull requests you merge are the ones that matter most. Autonomous remediation ships switched off, with every category set to Manual, so your team decides how much runs on its own and what still needs a human approval.
2. Gold Open Source: the safe version in seconds, free
gold.safeguard.sh is free and needs no login. Search any CVE or package to see affected and fixed versions, CVSS, EPSS, CISA KEV status and a recommended safe version, or use the "Am I affected?" checker. The same data is available through a free read API, README status badges and RSS threat feeds. The safeguard-sh/gold-scan-action GitHub Action can fail a build on a known exploited vulnerability, on any known vulnerability, or never, so a version you have just removed cannot quietly come back in next week's merge.
3. Gold Registry: start from components that are already fixed
registry.safeguard.sh provides hardened, drop in replacements for upstream packages and container images. Every Gold artifact is held continuously to zero exploitable critical or high CVEs, zero malware and a completed license check, with SLSA L3 or higher provenance, a Sigstore signature and a public evidence page you can hand to an auditor. An artifact loses its Gold badge if it regresses. Teams can point their own package directory at the registry so hardened equivalents resolve without changing how developers install anything.
When upstream has no fix yet, or will not ship one, Safeguard first works to get the fix accepted upstream. If the issue is severe or upstream will not move, Safeguard builds a fork, fixes the vulnerability there and publishes a hardened Gold build, so you are not stuck on a maintainer's release schedule. You can also request a hardened, compatibility tested build of a specific package. This fork and patch flow is in early access today, so ask which packages it covers before you plan around it.
| Your situation | What to use |
|---|---|
| A fixed version exists | Autonomous remediation opens the upgrade pull request; Gold Open Source confirms the safe version |
| You need to know today whether you are affected | Gold Open Source lookup and "Am I affected?" checker, free and without login |
| You want vulnerable versions to stop shipping | The Gold scan GitHub Action in CI, plus Safeguard policies and gates |
| Upstream has no fix yet, or will not ship one | A custom Gold request; fork and patch (early access) |
| You want to start clean on new projects | Pull hardened components and images from Gold Registry |
One honest limit: when the vulnerable code lives in a vendor appliance, a hosted service or an operating system runtime you do not build yourself, the fix still has to come from that vendor or distribution. The steps above apply to the open source dependencies and container images you build and ship, which is where most teams carry the largest and least tracked share of their exposure.
See how Safeguard would help
The first demo is a real Safeguard CLI session. You ask it to fix a lodash CVE in a repository; it reads the finding and the advisory, asks before each command, edits package.json to the fixed version, runs the install and the tests with your approval, and pushes nothing. The same flow applies to a camera app or an image processing service whose dependency tree carries a vulnerable codec or framework version.
Click through it at your own pace, or open it full screen.
The second demo sets up the local runner on your own machine, approves it with its device code, and then scans from the terminal. For an embedded Linux build with a camera stack, that is how you would scan the source tree and the container images you build on your own hardware.
Click through it at your own pace, or open it full screen.
The third demo connects Claude to the Safeguard MCP server with OAuth and lets you choose which Safeguard tools Claude may use. If your team already asks Claude about a camera pipeline's code, this lets it look up findings for those projects with permissions you set.
Click through it at your own pace, or open it full screen.
To try it on your own code, download Safeguard or read the CLI documentation.
Quick check
Why does a back side illuminated sensor need a second wafer?
Because the pixel wafer is thinned to a few micrometres so light can enter from the back, and silicon that thin cannot support itself. It is bonded face down to a support wafer first. In a stacked sensor that second wafer is a working logic wafer carrying the ADCs and processing.
Where is a sensor's defect pixel map stored, and why does it matter?
It is written during testing into one time programmable memory on the sensor or into the module's EEPROM. The ISP reads it to hide hot and dead pixels. If the map is lost or the wrong calibration data is used, bright speckles and color errors appear that look like hardware faults.
Sources
- Nobel Prize in Physics 2009, press release (Boyle and Smith, CCD)
- US Patent 3,971,065, Color imaging array (Bryce E. Bayer, Eastman Kodak)
- Fossum, "CMOS image sensors: electronic camera-on-a-chip", IEEE Transactions on Electron Devices 44(10), 1997, metadata
- Kagawa et al., "Novel stacked CMOS image sensor with advanced Cu2Cu hybrid bonding", IEDM 2016, metadata
- Haruta et al., "A 1/2.3inch 20Mpixel 3-layer stacked CMOS Image Sensor with DRAM", ISSCC 2017, metadata
- Stanford EE 392B, Lecture Notes 1: Silicon Photodetectors
- Stanford EE 392B, Lecture Notes 4: CMOS Image Sensors
- Sensor maker technology page: Back-illuminated Structure
- Sensor maker technology page: Stacked Structure
- Sensor maker technology page: Cu-Cu Connection
- EMVA 1288 standard
- MIPI Alliance: MIPI CSI-2 specification
- Linux kernel documentation: Camera sensor drivers
- Linux kernel documentation: V4L2 sub-devices
- Linux kernel documentation: Video for Linux API
- libcamera project
- Android Open Source Project: Camera
- Android Open Source Project: Privacy indicators
- Android Developers: Android 12 behavior changes for all apps
- Android Developers: Explain access to more sensitive information
- Apple Platform Security: Hardware microphone disconnect
- Brocker and Checkoway, "iSeeYou: Disabling the MacBook Webcam Indicator LED", USENIX Security 2014
Last reviewed: 23 October 2026. Sensor processes and interface specifications change; check the linked standards bodies and maker pages for current details.