Very fast C++ .PNG writer for 24/32bpp images.

fpng

Very fast C++ .PNG writer for 24/32bpp images. fpng.cpp was written to see just how fast you can write .PNG's without sacrificing too much compression, and to determine how slow the existing .PNG writers are. (Turns out they are quite slow.)

On average, fpng.cpp produces files roughly 1-15% smaller than QOI but is 1.4-1.5x slower on 24bpp images.

fpng.cpp compared to stb_image_write.h: fpng.cpp is over 10x faster with roughly 16-18% smaller files. (I was surprised by this. stb_image_write.h does a decent job choosing the PNG filter method and has a stronger byte-oriented parser than fpng's, but its Deflate compressor is crippled by being limited to only static Huffman tables.)

fpng.cpp compared to lodepng.cpp: fpng.cpp is 12-16x faster with roughly 15-18% larger files.

On 32bpp images with correlated alpha signals (a common case in video games) qoi outputs files 1.7x larger vs. fpng and 1.98x larger vs. lodepng.

Benchmark using the 184 QOI test images:

qoi:              4.69 secs   359.55 MB
fpng:             6.66 secs   353.50 MB
lodepng:          110.72 secs 300.14 MB
stb_image_write:  68.07 secs  425.64 MB

Benchmark using my 303 test images:

qoi:             2.25 secs  300.84 MB
fpng:            3.40 secs  253.26 MB
lodepng:         41.68 secs 220.40 MB
stb_image_write: 37.23 secs 311.41 MB

Benchmark using the 184 QOI test images, but with the green channel copied into alpha (to create a correlated alpha channel):

qoi:              2.75 secs   697.2 MB
fpng:             7.48 secs   404.0 MB
lodepng:          151.57 secs 352.1 MB
stbi:             81.58 secs  486.4 MB

182 of the qoi test images don't have alpha channels, so I'm guessing it's not being heavily tested with alpha images much yet.

Low-level description

fpng.cpp uses a custom image aware pixel-wise Deflate compressor which was optimized for simplicity over high ratios. It uses a simple LZRW1-style parser, all literals (except the PNG filter bytes) are output in groups of 3 or 4, all matches are multiples of 3/4 bytes, and it only utilizes a single dynamic Huffman block with one PNG IDAT block. It utilizes 64-bit registers and exploits unaligned little endian reads/writes. (On big endian CPU's it'll use 32/64bpp byteswaps.)

Passes over the input image and dynamic allocations are minimized, although it does use std::vector internally. The first scanline always uses filter #0, and the rest use filter #2 (previous scanline). It uses the fast CRC-32 code described by Brumme here. The original high-level PNG function (that code that writes the headers) was written by Alex Evans.

lodepng v20210627 fetched 12/18/2021

stb_image_write.h v1.16 fetched 12/18/2021

qoi.h fetched 12/18/2021

Building

To build, compile from the included .SLN with Visual Studio 2019/2022 or use cmake to generate a .SLN file. For Linux/OSX, use "cmake ." then "make". Tested with MSVC 2019/gcc/clang.

I have only tested fpng.cpp on little endian systems. The code is there for big endian, and it should work, but it needs testing.

Testing

From the "bin" directory, run "fpng_test.exe" or "./fpng_test" like this:

fpng_test.exe

To generate .CSV output only:

fpng_test.exe -c

There will be several output files written to the current directory: stbi.png, lodepng.png, qoi.qoi, and fpng.png. Statistics or .CSV data will be printed to stdout, and errors to stderr.

Using fpng

To use fpng.cpp in other programs, copy fpng.cpp/.h and Crc32.cpp/.h into your project. No other configuration or files are needed. Computing the CRC-32 of buffers is a substantial proportion of overall compression time in fpng, so if you have a faster CRC-32 function you can modify fpng_crc() in fpng.cpp to call that instead. The one included in Crc32.cpp doesn't utilize any special CPU instruction sets, so it could be faster.

#include "fpng.h" then call one of these C-style functions in the "fpng" namespace:

bool fpng_encode_image_to_memory(const void* pImage, int w, int h, int num_chans, bool flip, std::vector
   
    & out_buf);
bool fpng_encode_image_to_file(const char* pFilename, const void* pImage, int w, int h, int num_chans, bool flip);

   

num_chans must be 3 or 4. There must be w*3*h or w*4*h bytes pointed to by pImage. The image row pitch is always w*3 or w*4 bytes. There is no automatic determination if the image actually uses an alpha channel, so if you call it with 4 you will always get a 32bpp .PNG file.

Note the adler32 function in fpng.cpp could be vectorized for higher performance.

Fuzzing

I've started to fuzz fpng.cpp, but it will take several days to complete. So far it's survived several different random fuzz methods.

License

fpng.cpp/.h: Public Domain or zlib (your choice). See the end of fpng.cpp.

Crc32.cpp/.h: Copyright (c) 2011-2019 Stephan Brumme. zlib license:

https://github.com/stbrumme/crc32

Owner
Comments
  • Clean?

    Clean?

    :question: Maybe clean repo?

    $ git clone [email protected]:richgel999/fpng.git
    $ cd fpng
    $ git filter-branch --force --index-filter "git rm --cached --ignore-unmatch *.qoi" --prune-empty --tag-name-filter cat -- --all 
    WARNING: git-filter-branch has a glut of gotchas generating mangled history
             rewrites.  Hit Ctrl-C before proceeding to abort, then use an
             alternative filtering tool such as 'git filter-repo'
             (https://github.com/newren/git-filter-repo/) instead.  See the
             filter-branch manual page for more details; to squelch this warning,
             set FILTER_BRANCH_SQUELCH_WARNING=1.
    Proceeding with filter-branch...
    
    Rewrite 9fad7d93d9b23ade59276a9e25ecaf8be45b3dc5 (2/29) (0 seconds passed, remaining 0 predicted)    rm 'qoi.qoi'
    Rewrite 5f3d874a6d838f668bd01a1c9bacf909455f3ba5 (3/29) (0 seconds passed, remaining 0 predicted)    rm 'qoi.qoi'
    Rewrite 4934bd832f66c0d83e74724ef74f41352086033d (4/29) (0 seconds passed, remaining 0 predicted)    rm 'qoi.qoi'
    Rewrite ad003486b32f2a2de02bd7bf27f7929c77d77450 (5/29) (0 seconds passed, remaining 0 predicted)    rm 'qoi.qoi'
    Rewrite 99d8f93240a24e62bab83d6d9ac7b14899d03141 (6/29) (0 seconds passed, remaining 0 predicted)    rm 'qoi.qoi'
    Rewrite 3d5e2598a3a95b61ff812a7409fa7b3a8f0990cf (7/29) (0 seconds passed, remaining 0 predicted)    rm 'qoi.qoi'
    Rewrite af180bffa43cbdbeb8cebc400e799468a0eb3d2d (8/29) (0 seconds passed, remaining 0 predicted)    rm 'qoi.qoi'
    Rewrite fb46604b56652f1313cd0c93d8be3a7645be29a8 (9/29) (0 seconds passed, remaining 0 predicted)    rm 'qoi.qoi'
    Rewrite 83b54820e0f0e82bbadfcd9d81fcfbe5b629f599 (10/29) (0 seconds passed, remaining 0 predicted)    rm 'qoi.qoi'
    Rewrite ce637ad28caa147232916e905b13930e1ba504e9 (11/29) (0 seconds passed, remaining 0 predicted)    rm 'qoi.qoi'
    Rewrite 540c2b5f4bd4004c33aa71d5ee90433c786fe1ea (12/29) (0 seconds passed, remaining 0 predicted)    rm 'qoi.qoi'
    Rewrite b75f2290027350f7e4cb7d0cf063e94ad3dc09ac (13/29) (0 seconds passed, remaining 0 predicted)    rm 'qoi.qoi'
    Rewrite c1953aff389dbda4cfef43443fe1edb43a293701 (14/29) (0 seconds passed, remaining 0 predicted)    rm 'qoi.qoi'
    Rewrite 3292ceaf0bc599c5c13394a65548364a1642266c (15/29) (0 seconds passed, remaining 0 predicted)    rm 'qoi.qoi'
    Rewrite 1f62a7ae4d43cf412fa90132752643d8dc373f9f (16/29) (0 seconds passed, remaining 0 predicted)    rm 'qoi.qoi'
    Rewrite f55421f1e9b46610d9d9914c61ea5460a3cc699c (17/29) (0 seconds passed, remaining 0 predicted)    rm 'qoi.qoi'
    Rewrite a011faa73413f82c82ce283927eebac97f929f0f (18/29) (0 seconds passed, remaining 0 predicted)    rm 'qoi.qoi'
    Rewrite 009ab0deca2ed74b7b69cfea5ecb7dbe62e91cf8 (19/29) (0 seconds passed, remaining 0 predicted)    rm 'qoi.qoi'
    Rewrite 8b4515342a12a1784db143a9db647881f31007b4 (20/29) (0 seconds passed, remaining 0 predicted)    rm 'qoi.qoi'
    Rewrite ff60a4fd6c9af8510417bcba569738d8991a2cb0 (21/29) (0 seconds passed, remaining 0 predicted)    rm 'qoi.qoi'
    Rewrite a1864802167782f700bd15ba3b4c497cc978af07 (22/29) (0 seconds passed, remaining 0 predicted)    rm 'qoi.qoi'
    Rewrite 2f10f60f9c842f4bd5645b84ead836a9ec26a142 (23/29) (0 seconds passed, remaining 0 predicted)    rm 'qoi.qoi'
    Rewrite 782059869438bbf16e1cce2f012409e3bfed172a (24/29) (0 seconds passed, remaining 0 predicted)    rm 'qoi.qoi'
    Rewrite a651b054896d862934a3bd79e78cb770ec14d11c (25/29) (0 seconds passed, remaining 0 predicted)    rm 'qoi.qoi'
    Rewrite 6b9ca2755f15c40ce54025c31c9f94af431f2763 (26/29) (0 seconds passed, remaining 0 predicted)    rm 'qoi.qoi'
    Rewrite b12cb639087e5527b815fefde0a04e58fcf58bfc (27/29) (0 seconds passed, remaining 0 predicted)    rm 'qoi.qoi'
    Rewrite 29f97115cc90872fe5bae43cbdc8213a53abf42b (29/29) (0 seconds passed, remaining 0 predicted)    
    Ref 'refs/heads/main' was rewritten
    Ref 'refs/remotes/origin/main' was rewritten
    WARNING: Ref 'refs/remotes/origin/main' is unchanged
    Ref 'refs/tags/initial' was rewritten
    $ git gc --prune=now --aggressive
    $ git repack -a -d -f --depth=250 --window=250
    $ git shortlog -s -n
    $ git push origin --force --all
    $ git push origin --force --tags
    

    Current repo size: 528.00KiB. Expected repo size: 528 - 333(qoi.qoi) = 195.00KiB

    PS: I can do this on my fork for demonstration. Need to?

  • Compilation fails on Apple M1

    Compilation fails on Apple M1

    I don't manage to compile fpng on my Apple MacBook Pro M1 (ARM). I get the following compilation error:

    fpng.cpp:304:22: error: invalid output constraint '+b' in asm
                    __asm__("cpuid;" : "+b"(ebx), "+a"(eax), "+c"(ecx), "=d"(edx));
    

    System:

    $ uname -a
    Darwin macbookpro 21.2.0 Darwin Kernel Version 21.2.0: Sun Nov 28 20:28:41 PST 2021; root:xnu-8019.61.5~1/RELEASE_ARM64_T6000 arm64
    
    /Library/Developer/CommandLineTools/usr/bin/c++ --version
    Apple clang version 13.0.0 (clang-1300.0.29.30)
    Target: arm64-apple-darwin21.2.0
    Thread model: posix
    InstalledDir: /Library/Developer/CommandLineTools/usr/bin
    

    Any help appreciated! Thanks.

  • Request a valid version number tag / release.

    Request a valid version number tag / release.

    fpng is great, I am adding it to Xmake's package management repository, it requires a valid version number to do the version of the package, and then download the specified tar.gz through the version number. like this, v1.0, v1.0.1, ...

    see https://github.com/xmake-io/xmake-repo/pull/812

    Although even if there is no version number, I can also implement it through git commit + fake version, but this is not recommended.

    I hope that fpng can have a good version tag list, thank you very much.

  • Cannot load my png-file...

    Cannot load my png-file...

    Hello!

    I'm trying to add FPNG to our image error metric called FLIP (https://github.com/NVlabs/flip/) because we need faster PNG loading. Unfortunately, I cannot load the first image I tried with: reference

    The code exits on line 3033 in fpng.cpp: if ((idat_ofs) || (!found_fdec_chunk)) return FPNG_DECODE_NOT_FPNG;

    The cause is that found_fdec_chunk==false. Any ideas on what can be wrong? The image can be loaded by stbi_image.h and PhotoShop.

    Thanks in advance! /Tomas

  • Not an Issue - Just Need Help

    Not an Issue - Just Need Help

    Hey! So I am I'm new to all of this in terms of C. I am just a web dev and trying to figure out how to maybe compress PNG images to be usable (Size of File for Speed) on web. Was looking at this and QOI. But this one looks a lot more promising due to it being compatible with PNG decoders already. Anyway, I am doing the Testing section and trying to convert my PNG to FPNG but I am just getting a bigger file of my original PNG or the same size.

    Am I do doing it right?

    // CMD Line in ./fpng_main/bin
    
    ./fpng.exe -s test.png
    

    Is there something else I need to be doing? Sorry again, I am a complete newbie to this type of stuff lol

  • Possible (likely innocuous) typo in chunk name parsing

    Possible (likely innocuous) typo in chunk name parsing

    I was looking over the source code, and I noticed this oddity in the code for detecting chunk type

    https://github.com/richgel999/fpng/blob/b964f657bd3ace341df47fab3c249a24c6eac995/src/fpng.cpp#L2520-L2526

    for (uint32_t i = 0; i < 4; i++)
    {
    	const uint8_t c = pChunk->m_type[0];
    

    It appears that it's checking for non-alphabetical characters in the chunk type string, but it's only repeatedly checking that that's true for the first character in the type. I suspect m_type[0] was meant to be m_type[i]

    I don't believe that this would cause problems, since the other checks should still reject any not-recognized chunk types

  • QOI has a more comprehensive test suite

    QOI has a more comprehensive test suite

    fpng's top-level README.md says "the [emphasis added] 184 QOI test images (note 182 of the qoi test images don't have alpha channels, so this is almost entirely a 24bpp test)".

    QOI updated its test suite a month ago (https://github.com/phoboslab/qoi/issues/69). The new suite contains 2848 images.

  • Update README.md with Wuffs numbers

    Update README.md with Wuffs numbers

    There's lodepng and stb_image numbers, but Wuffs is faster than both.

    https://github.com/randy408/libspng is another "single file C library" contender, if you haven't already seen it.

  • Python bindings for fpng

    Python bindings for fpng

    Hi, thanks for the great library. I had a need to use this from python, so I wrote python bindings that seem to have a good speedup over opencv-python's imwrite/imencode.

    I published the interface here, with the same license as this project: https://github.com/qrmt/fpng-python

    Perhaps someone else will find find use for the bindings (and possibly review it), so I thought I'd post it here. Cheers!

  • Adding resolution/icc info tags and 8 bits support ?

    Adding resolution/icc info tags and 8 bits support ?

    Hi Rich, (I sent an email about this issue, but I guess it didn't reach you)

    fpng is really efficient and interesting work. Congratulations !

    However, to cover most of png usual features (and according my need), I think it would be great to add the following support/features :

    • resolution information (easy to add)
    • ICC profile (or at least the sRGB tag) (somehow easy to add)
    • support for indexed pictures (more complicated)

    I think that I can add the support of resolution and icc profile, keeping the code fast and the API simple (just passing an extra pointer for optional tags, that can be extend to other tags in the future)...

    Do you think you may extend your code in the future to support indexed images ?

    Best regards

    Sébastien Léon

  • Simplify wuffs_decode implementation

    Simplify wuffs_decode implementation

    Wuffs has a low-level C API, necessary if e.g. decoding an animated image (or decoding incrementally over the network) while concurrently doing other work, and a high-level C++ API. fpng_test just wants to decode a still (non-animated) PNG image whose compressed form is entirely in memory, so it can use the simpler C++ API.

  • Ignore checksums more often

    Ignore checksums more often

    Adler-32 and CRC-32 computations are fast with SIMD, but if you just want the fastest possible PNG decoder (e.g. to compare to QOI's speed), ignoring the checksums can be even faster.

    For decode (not encode, obviously), IIUC fpng already skips computing/verifying Adler-32 always and CRC-32 sometimes (depending on FPNG_DISABLE_DECODE_CRC32_CHECKS).

    Some of the other libraries (lodepng and stb_image) also do this automatically. Wuffs needs to opt in, with a one-liner patch:

    diff --git a/src/fpng_test.cpp b/src/fpng_test.cpp
    index afc3f77..7302146 100644
    --- a/src/fpng_test.cpp
    +++ b/src/fpng_test.cpp
    @@ -683,6 +683,7 @@ static void* wuffs_decode(void* pData, size_t data_len, uint32_t &width, uint32_
            wuffs_png__decoder* pDec = wuffs_png__decoder__alloc();
            if (!pDec) 
                    return nullptr;
    +       wuffs_png__decoder__set_quirk_enabled(pDec, WUFFS_BASE__QUIRK_IGNORE_CHECKSUM, true);
     
            wuffs_base__image_config ic;
            wuffs_base__io_buffer src = wuffs_base__ptr_u8__reader((uint8_t *)pData, data_len, true);
    
Fast streaming PNG<->QOI converter with some compression-improving extensions

QOIG Fast streaming PNG<->QOI converter with some compression-improving extensions. Can achieve 1%-10% better compression than QOI without sacrificing

Oct 3, 2022
CGIF, A fast and lightweight GIF encoder that can create GIF animations and images

CGIF, a GIF encoder written in C A fast and lightweight GIF encoder that can create GIF animations and images. Summary of the main features: user-defi

Dec 28, 2022
Arduino PNG image decoder library
Arduino PNG image decoder library

An 'embedded-friendly' (aka Arduino) PNG image decoding library

Jan 6, 2023
pngtostl is a program that converts a PNG image into STL 3D models
pngtostl is a program that converts a PNG image into STL 3D models

pngtostl is a program that converts a PNG image into a litophane, in STL format, suitable to be printed by entry level 3D printers.

Dec 17, 2022
libspng is a C library for reading and writing PNG format files with a focus on security and ease of use.
libspng is a C library for reading and writing PNG format files with a focus on security and ease of use.

libspng (simple png) is a C library for reading and writing Portable Network Graphics (PNG) format files with a focus on security and ease of use.

Dec 29, 2022
Simple, generally spec-compliant, hacky PNG Decoder written in C99.

Simple, generally spec-compliant, hacky PNG Decoder written in C99.

Nov 2, 2021
An image and texture viewer for tga, png, apng, exr, dds, gif, hdr, jpg, tif, ico, webp, and bmp files
An image and texture viewer for tga, png, apng, exr, dds, gif, hdr, jpg, tif, ico, webp, and bmp files

An image and texture viewer for tga, png, apng, exr, dds, gif, hdr, jpg, tif, ico, webp, and bmp files. Uses Dear ImGui, OpenGL, and Tacent. Useful for game devs as it displays information like the presence of an alpha channel and querying specific pixels for their colour.

Dec 31, 2022
PNG encoder and decoder in C and C++.

PNG encoder and decoder in C and C++, without dependencies

Dec 25, 2022
Rate-Distortion Optimized Lossy PNG Encoding Tool

rdopng is a command line tool which uses LZ match optimization, Lagrangian multiplier rate distortion optimization (RDO), a simple perceptual error tolerance model, and Oklab-based colorspace error metrics to encode 24/32bpp PNG files which are 30-80% smaller relative to lodepng/libpng.

Dec 24, 2022
An open source library for face detection in images. The face detection speed can reach 1000FPS.
An open source library for face detection in images. The face detection speed can reach 1000FPS.

libfacedetection This is an open source library for CNN-based face detection in images. The CNN model has been converted to static variables in C sour

Jan 8, 2023
Reading, writing, and processing images in a wide variety of file formats, using a format-agnostic API, aimed at VFX applications.

README for OpenImageIO Introduction The primary target audience for OIIO is VFX studios and developers of tools such as renderers, compositors, viewer

Jan 2, 2023
HDRView is a simple research-oriented image viewer with an emphasis on examining and comparing high-dynamic range (HDR) images
HDRView is a simple research-oriented image viewer with an emphasis on examining and comparing high-dynamic range (HDR) images

HDRView is a simple research-oriented high-dynamic range image viewer with an emphasis on examining and comparing images, and including minimalistic tonemapping capabilities. HDRView currently supports reading EXR, PNG, TGA, BMP, HDR, JPG, GIF, PNM, PFM, and PSD images and writing EXR, HDR, PNG, TGA, PPM, PFM, and BMP images.

Jan 5, 2023
A simple API wrapper that generates images & facts of any animal

animality.h A simple API wrapper that generates images & facts of any animal Required dependencies: libcurl for sending HTTPS requests. pthreads for t

Nov 10, 2022
You can use this to compile the code and output images into a word doc for assignment purposes
You can use this to compile the code and output images into a word doc for assignment purposes

Code_n_Ouput_to_docx You can use this to compile the code and output images into a word doc for assignment purposes Basic requirements: Python 3.7 or

Aug 21, 2022
A crude untested example showing how to retrieve and display images from multiple cameras with OpenCV and wxWidgets.

About wxOpenCVCameras is a crude untested example of how to retrieve and display images from multiple cameras, using OpenCV to grab images from a came

Dec 14, 2022
Convert images to ASCII art.
Convert images to ASCII art.

Image-to-ascii Convert images to ASCII art. This program using stb library to load images. Usage Usage: compile the program to get *.out file

Aug 20, 2022
Make It Pixel is a programming language to process images to look like pixel art.
Make It Pixel is a programming language to process images to look like pixel art.

Make images look like pixel art Make It Pixel is a programming language to process images to look like pixel art. Its interpreter is written in C++ an

Nov 24, 2022
Given a set of images, LeastAverageImage generates the least average -- the opposite of the average.
Given a set of images, LeastAverageImage generates the least average -- the opposite of the average.

LeastAverageImage Given a set of images, LeastAverageImage generates the least average -- the opposite of the average. More information about the prog

Nov 27, 2022
A (very) simple notification wrapper for Dear ImGui
A (very) simple notification wrapper for Dear ImGui

imgui-notify Is a header-only wrapper made to create notifications with Dear ImGui. As I couldn't find any library for this I just decided to create m

Jan 2, 2023