CNN Digit Detector | Al Hossain Shawn

A complete end-to-end pipeline that trains a 2D Convolutional Neural Network in PyTorch, exports its weights as fixed-point C++ constants, synthesises the network logic into hardware via Vitis HLS, and deploys the resulting bitstream to an Artix-7 FPGA, displaying live digit recognition over HDMI.

Target Device	PA-StarLite · Artix-7 XC7A200T (215 K LUTs · 740 DSPs · 13 Mb BRAM)
Toolchain	Vitis HLS · AMD Vivado · PyTorch · Python (sklearn)
Dataset	scikit-learn Digits containing 1,797 handwritten images (8×8 px, classes 0–9)
Interface	Push-buttons · UART debug · HDMI live display (TMDS · 200 MHz diff. clock)
Arithmetic	`ap_fixed<16,6>` (16-bit with 6 integer bits and 10 fractional bits)
Clock	200 MHz differential (DIFF_SSTL135, 5.0 ns period)
Dynamic Power	0.405 W total (DSP 0.074 W · MMCM 0.146 W · I/O 0.137 W)

Live Demonstration

The video below shows the CNN running on hardware, where push-buttons cycle through the stored digit dataset, the 8×8 pixel image is rendered on the HDMI monitor, and the predicted class is displayed in real time alongside the 10-class logit bar chart.

Live on PA-StarLite · Artix-7 XC7A200T · 200 MHz

What you are seeing

The CNN bitstream is running directly on FPGA hardware. Physical push-buttons cycle through 1,797 stored digit images. For each button press:

The selected 8×8 pixel image is fetched from on-chip BRAM and fed to the CNN accelerator.
The hardware completes a full forward pass (Conv → ReLU → MaxPool → FC) in a single clock burst at 200 MHz.
The HDMI output renders the pixel grid (left panel) and a 10-class logit bar chart side-by-side on the monitor.
The tallest bar identifies the predicted digit class in real time, with no CPU involved after the AXI handshake.

Video is muted and loops automatically. Use the controls to pause or scrub.

PA-StarLite FPGA board running the CNN digit detector — PA-StarLite development board (Artix-7 XC7A200T) with HDMI and UART cables connected, running the synthesised CNN bitstream.

CNN Hardware Architecture

The forward pass is a three-layer network: a 3×3 convolution with ReLU activation, a 2×2 max-pooling, and a fully-connected linear classifier for 10 digit classes. All intermediate buffers are dual-port BRAMs partitioned across channel dimensions for II=1 pipelining. In HLS, each block becomes physical hardware, including multipliers, adders, comparators, and Block RAMs, all operating concurrently.

CNN Hardware Pipeline

flowchart LR classDef io fill:#1a2744,stroke:#303e66,stroke-width:2px,color:#fff,font-weight:bold classDef layer fill:#1e5f74,stroke:#2d8da8,stroke-width:2px,color:#fff classDef act fill:#7a4100,stroke:#c06000,stroke-width:2px,color:#fff classDef buf fill:#1a4731,stroke:#28a262,stroke-width:2px,color:#fff classDef out fill:#4a1942,stroke:#9333ea,stroke-width:2px,color:#fff IN(["Input\n1×8×8"]):::io CONV["Conv 3×3\n4 filters · ReLU\nBias ROM"]:::layer CBUF[("conv_out\n4×8×8")]:::buf POOL["MaxPool 2×2\nStride 2\nComparator tree"]:::layer PBUF[("pool_out\n4×4×4")]:::buf FC["Flatten → FC\n64→10 dot products\nWeight ROMs"]:::layer OUT(["Prediction\n10 logits"]):::out IN -->|"1×8×8"| CONV CONV -->|"4×8×8"| CBUF CBUF -->|"4×8×8"| POOL POOL -->|"4×4×4"| PBUF PBUF -->|"64 values"| FC FC -->|"argmax"| OUT

CNN hardware dataflow showing three pipelined stages. BRAM buffers (cylinders) hold intermediate feature maps, and all loops run at II=1 with full channel unrolling.

HLS Design

The CNN function is described in C++ with HLS pragmas. Key optimisations: full channel-loop unrolling (4 parallel convolution engines) and complete array partitioning on the BRAM buffers so all four channels can be read or written in a single cycle.

HLS Compilation Flow

flowchart LR classDef sw fill:#374151,stroke:#6b7280,color:#fff classDef hls fill:#1a2744,stroke:#4a5d8c,color:#fff,font-weight:bold classDef pragma fill:#1e5f74,stroke:#2d8da8,color:#fff classDef hw fill:#1a4731,stroke:#28a262,color:#fff classDef fpga fill:#4a1942,stroke:#9333ea,color:#fff PY["PyTorch\ntrain_cnn.py"]:::sw WH["weights.h\nap_fixed constants"]:::sw CPP["cnn.cpp + cnn.h\nHLS C++ description"]:::hls PRAGS["HLS Pragmas\nUNROLL · PIPELINE II=1\nARRAY_PARTITION dim=1"]:::pragma VLG["cnn.v RTL\nDSP48 · BRAM · ROMs"]:::hw BIT["Bitstream\nPA-StarLite 200 MHz"]:::fpga PY -->|"export"| WH WH --> CPP CPP --> PRAGS PRAGS -->|"synthesise"| VLG VLG -->|"impl + P&R"| BIT

HLS compilation flow: PyTorch trains the model and exports fixed-point weights; Vitis HLS synthesises the C++ forward pass to RTL; Vivado places, routes, and generates the bitstream.

// Channel loop unrolled → 4 parallel hardware convolution engines
conv_label_oc: for (int oc = 0; oc < 4; oc++) {
    #pragma HLS UNROLL
    for (int r = 0; r < 8; r++) {
        for (int c = 0; c < 8; c++) {
            #pragma HLS PIPELINE II=1
            // 3×3 kernel accumulation with zero-padding
        }
    }
}

data_t conv_out[4][8][8];
#pragma HLS ARRAY_PARTITION variable=conv_out complete dim=1  // all 4ch in parallel

RTL Architecture

The HLS synthesis generates a Verilog hierarchy under cnn.v, orchestrated by a 20-state Control FSM that schedules overlapping convolution and pooling to maximise utilisation: one filter pair convolved while the previous is being pooled.

RTL Module Hierarchy & FSM

flowchart TD classDef top fill:#1a2744,stroke:#4a5d8c,stroke-width:3px,color:#fff,font-weight:bold classDef pipe fill:#1e5f74,stroke:#2d8da8,stroke-width:2px,color:#fff classDef mem fill:#1a4731,stroke:#28a262,stroke-width:2px,color:#fff classDef math fill:#4a1942,stroke:#9333ea,stroke-width:2px,color:#fff classDef fsm fill:#374151,stroke:#6b7280,stroke-width:2px,color:#fff CNN["cnn.v: Top Level\n20-state Control FSM + Bus MUX"]:::top CNN --> CRAM["8× conv_out BRAM\nDual-port · 4 channels"]:::mem CNN --> PRAM["4× pool_out BRAM\nDual-port · 4 channels"]:::mem CNN --> CPIPE["4× Conv Pipeline\nRow/Col loop · DSP MAC"]:::pipe CNN --> PPIPE["4× Pool Pipeline\nComparator tree · 2×2 window"]:::pipe CNN --> FCPIPE["FC Classifier Pipeline"]:::pipe FCPIPE --> WROM["64× Weight ROMs\nfc_w_0 … fc_w_63"]:::mem FCPIPE --> BROM["1× Bias ROM\nfc_b"]:::mem FCPIPE --> MACARR["DSP48 MAC Array\ncnn_mac_muladd_..."]:::math subgraph FSM ["FSM Execution Schedule"] ST1["S1: Idle, wait ap_start"]:::fsm ST2["S2–10: CONV + POOL\nOverlapping pipelines"]:::fsm ST3["S11–19: Register load\n8 pool values / cycle"]:::fsm ST4["S20: FC pipeline\nAssert ap_done"]:::fsm ST1 --> ST2 --> ST3 --> ST4 --> ST1 end

RTL hierarchy generated by Vitis HLS. The FSM (inset) overlaps convolution and pooling in States 2–10, pre-loads FC registers in States 11–19, then runs 10 dot products in parallel at State 20.

Hardware & Results

HDMI monitor showing digit recognition output — HDMI display output showing the 8×8 input pixel grid (left box) and the 10-class logit bar chart rendered on the monitor by the Artix-7 fabric in real time.

Vivado device floor plan after place and route — Vivado device floor plan: CNN logic placed in the lower pblock regions (X0Y0–X1Y2), with MMCM and I/O logic visible in upper banks.

On-chip power analysis from Vivado — Vivado on-chip power analysis: 0.405 W dynamic total. MMCM (36%) and I/O (33%) dominate due to the 200 MHz HDMI TMDS differential output.

Vivado elaborated schematic: `cnn.v` top-level wrapper showing ROM, BRAM, DSP MAC block connections, and HDMI output stage.

Implementation Summary

Metric	Value
Clock Frequency	200 MHz (timing closure achieved, no negative slack)
Pipeline II	1 (both convolution and pooling spatial loops)
FSM States	20 (overlapping CONV/POOL + parallel register pre-load)
Dynamic Power	0.405 W (MMCM 0.146 W · I/O 0.137 W · DSP 0.074 W)
Arithmetic	ap_fixed<16,6> (bit-accurate co-simulation matches PyTorch FP32 outputs)
Weight storage	64 fc_w ROMs + 1 fc_b ROM + conv_w/conv_b constants in LUT fabric