filter multipixel video streams -凯发k8网页登录

multipixel streaming using library blocks

generate a multipixel stream from the frame to pixels block by setting number of pixels to 4 or 8. the default value of 1 returns a scalar pixel stream with a sample rate of total pixels per line * total video lines faster than the frame rate. this rate shows red in the example model. the two multipixel subsystems use a multipixel stream with number of pixels set to 4. this configuration returns 4 pixels on each clock cycle and has a sample rate of (total pixels per line/4) * total video lines. the lower output rate, which is green in the model, shows that you can increase either the input frame rate or resolution by a factor of 4 and therefore process 4 times as many pixels in the same frame period using the same clock frequency as the single pixel case.

the singlepixelgaussianedge and multipixelgaussianedge subsystems compute the same result using the image filter and edge detector blocks.

in multipixelgaussianedge, the blocks accept and return four pixels on each clock cycle. you do not have to configure the blocks for multipixel streaming, they detect the input size on the port. the pixelcontrol bus indicates the validity and location in the frame of each set of four pixels. the blocks buffer the [4x1] stream to form four [ kernelheight x kernelwidth ] kernels, and compute four convolutions in parallel to give a [4x1] output.

custom multipixel algorithms

the multipixelcustomgaussianedge subsystem uses the line buffer block to implement a custom filtering algorithm. this subsystem is similar to how the library blocks internally implement multipixel kernel operations. the image filter and edge detector blocks use more detailed optimizations than are shown here. this implementation shows a starting point for building custom multipixel algorithms using the output of the line buffer block.

the custom filter and custom edge detector use the line buffer block to return successive [ kernelheight x numberofpixels ] regions. each region is passed to the kernelindexer subsystem which uses buffering and indexing logic to form number of pixels * [ kernelheight x kernelwidth ] filter kernels. then each kernel is passed to a separate filterkernel subsystem to perform convolutions in parallel.

form kernels from line buffer output

the kernelindexer subsystem forms 4 [5x5] filter kernels from the 2-d output of the line buffer block.

the diagram shows how the filter kernel is extracted from the [5x4] output stream, for the kernel that is centered on the first pixel in the [4x1] output. this first kernel includes pixels from 2 adjacent [5x4] line buffer outputs.

the kernel centered on the last pixel in the [4x1] output also includes the third adjacent [5x4] output. so, to form four [5x5] kernels, the subsystem must access columns from three [5x4] matrices.

the kernelindexer subsystem uses the current [5x4] input, and stores two more [5x4] matrices using registers enabled by shiftenable. this design is similar to the tapped delay line used with a line buffer using single pixel streaming. the subsystem then accesses pixel data across the columns to form the four [5x5] kernels. the image filter block uses this same logic internally when the block has multipixel input. the block automatically designs this logic at compile time for any supported kernel size.

implement filters

since the input multipixel stream is a [4x1] vector, the filters must perform four convolutions on each cycle to keep pace with the incoming data. there are four parallel filterkernel subsystems that each perform the same operation. the [5x5] matrix multiply is implemented as a [5x1] vector multiply by using a for each subsystem containing a pipelined multiplier. the output is passed to an adder tree. the adder tree is also pipelined, and the pipeline latency is applied to the pixelcontrol signal to match. the results of the four filterkernel subsystems are then concatenated into a [4x1] output vector.

implement edge detectors

to match the algorithm of the edge detector block, this custom edge detector uses a [3x3] kernel size. compare this kernelindexer subsystem for the [3x3] edge detection with the [5x5] kernel described above. the algorithm still must access three successive matrices from the output of the line buffer block (including padding on either side of the kernel). however, the algorithm saves fewer columns to form a smaller filter kernel.

extending to larger kernel sizes

for larger kernel sizes the number of [ kernelheight x numpixels ] regions to store in the kernelindexer is (2 * ceil(floor(kernelwidth / 2) / numpixels) 1). in such a case, the number of inputs to the concatenators increases to kernelwidth and you must route these additional inputs from the tapped delay line of line buffer matrices. for a [4x1] multipixel stream with a [11x11] kernel size you would need to store five [11x4] matrices from the line buffer to form four [11x11] kernels each cycle.

improving simulation time

in the default example configuration, the single pixel, multipixel, and custom multipixel subsystems all run in parallel. the simulation speed is limited by the time processing the single-pixel path because it requires more iterations to process the same size of frame. to observe the simulation speed improvement for multipixel streaming, comment out the single-pixel data path.

hdl implementation results

hdl was generated from both the multipixelgaussianedge subsystem and the multipixelcustomgaussianedge subsystem and put through place and route on a xilinx™ zc706 board. the multipixelcustomgaussianedge subsystem, which does not attempt to optimize coefficients, had the following results -

t =
  4x2 table
    resource     usage
    _________    _____
    dsp48        108  
    flip flop    9842 
    lut          4960 
    bram         12   

the multipixelgaussianedge subsystem, which uses the optimized image filter and edge detector blocks uses less resources, as shown in the table below. this comparison shows the resource savings achieved because the blocks analyze the filter structure and pre-add repeated coefficients.

t =
  4x2 table
    resource     usage
    _________    _____
    dsp48        16   
    flip flop    3959 
    lut          1789 
    bram         10