PNG is an image format that has a history of development beginning in 1995, and it is still a popular, long living format. Generally, it is known for its features such as lossless compression and the ability to handle transparent pixels.
However, we do not look at image formats from a general point of view, but rather think of ways to glitch them. When we look at PNG from the point of view of glitch, what kind of peculiarity does it have?
We should first look into the checksum system of the CRC32 algorithm. It is used to confirm corrupted images, and when it detects corruption in an image file, normal viewer applications refuse to display it. Therefore, it is impossible to generate glitches using simple methods such as rewriting part of the binary data using text editors or binary editors (you will completely fail). In other words, the PNG format is difficult to glitch.
We need to create glitches accordingly to the PNG specification in order to avoid this failure. This means that we must rewrite the data after decoding CRC32, re-calculate it and attach it to the edited data.
Next we want to look at the transcode process of PNG. The chart shown below is a simplified explanation of how PNG encoding flows.
Each of the four states that are shown above can be glitch targets. However, glitching the the first “Raw Data” is the same as glitching BMP, so it technically isn’t a PNG glitch (at the end, it is the same as PNG with the None filter applied. I will explain this in the next section). The final “Formatted PNG” glitch will not work because of the checksum system I mentioned above.
This means that PNG glitches can be made when the “Filtered Data” or “Compressed Data” is manipulated. I will explain about filters in the following subsection. When “Filtered Data” is glitched, it shows a distinctive effect; patterns that look like flower petals scatter around the image. The difference between the filters become clear when the “Filtered Data” is glitched. On the other hand, “Compressed Data” glitches are flavored by their own compression algorithm, which is Deflate compression. It shows an effect similar to a snow noise image.
There are elements else besides the transcoding process that could also influence the appearance of glitches such as transparent pixels and interlaces.
The factor that characterizes the appearance of glitches the most is the process called filter. The filter converts the uncompressed pixel data of each scanline using a certain algorithm in order to improve the compression efficiency. There are five types of filters that include four algorithms called Sub, Up, Average and Paeth, and also None (which means no filter applied). PNG images are usually compressed after the most suitable filter is applied to each scanline, and therefore all five filters are combined when PNG images are made.
These five filters usually only contribute to the compression efficiency, so the output result is always the same no matter which filter is applied. However, a clear difference appears in the output result when the filtered data is damaged. It is difficult to recognize the difference of the filters when an image is optimized and has all five filters combined, but the difference becomes obvious when an image is glitched when the same, single filter is applied to each scanline.
I will show the difference of the effect that each filter has later on, but when we look close into the results, we will understand which filter is causing which part of the beauty of PNG glitches (yes, they are beautiful) to occur.
I will show the actual glitch results in the next section.
I have shown two PNG images above: one is an image before it has been glitched, and one is an image that has been glitched.
This is a Filtered Data glitch, which I explained in the previous section.
The original PNG has optimized filters applied to each scanline, and all of the five filters have been combined. The glitch reveals how the five filters were balanced when they were the combined.
Lets look into the difference between each filter type.
The image above has applied “None (no filter)”, meaning that it is a raw data glitch. Each pixel stands alone in this state and do not have any relationship with the others, so a single re-wrote byte does not have a wide range influence.
This is a glitched image that has the filter “Sub” applied to each scanline. When the Sub algorythm is applied, the target pixel rewrites itself by refering to the pixel that is right next to it. This is why the glitch pattern avalanches towards the right side.
This is the filter “Up”. This filter is similar to Sub, but its reference direction is the top and bottom.
The filter “Average” refers to a diagonal direction. It shows a meteor like tail that starts from the damaged pixel. The soft gradation effect is also one of the peculiarities of this filter. The result of a PNG glitch when the Average filter is applied is a glitch that lacks glitchiness, and is also the most delicate portion of PNG glitching.
The filter “Paeth” has the most complicated algorithm when compared with the others. It also has the most complicated glitch effect. The glitch will affect a wide range of areas even with the least byte re-writing. The keynote effect of PNG glitch is caused by this filter; the figure shown in the original image is maintained, but is intensely destroyed at the same time.
This is a glitch of the state that I referred to as Compressed Data in the previous section. A snowstorm effect appears, and it is difficult to recognize the original figure in the image. It infrequently remains to show effects of the filters. The image is often completely destroyed.
Lets look into what happens when an image that includes transparent pixels is glitched.
The transparency comes as an effect. Especially the filter “Average” seems to blend transparent pixels gradually.
A 100% gathering of transparent pixels is handled in the same way as a solid colored section. You can tell that the filter “Up” is often applied to solid colored sections.
(There is a possibility that newer general-purpose image formats switch their compression scheme of each part depending on if the image is a solid colored section, or else a complicated image such as photographs. The use of images that include solid colored sections for testing glitches is an effective method. One example is a WebP. )
PNG interlaces are divided into seven passes, using the Adam7 algorithm based on 8x8 pixels. We are able to visualy observe that algorithm when an interlaced PNG is glitched. We can also confirm a stitched effect, and that its angle has become narrow towards the Average filter (see appendix B).
PNG is a very simple format compared to JPEG or other new image formats. The filter algorithms are like toys, and its compression method is the same as oldschool Zip compression. However, this simple image format shows a surprisingly wide range of glitch variations. We would perhaps only need one example to explain a JPEG glitch, but we need many different types of samples in order to explain what a PNG glitch is.
PNG was developed as an alternative format of GIF. However, when it comes to glitching, GIF is a format that is too poor to be compared with PNG. PNG has prepared surprisingly rich results that have been concealed by the checksum barrier for a long time.
The author released a tiny script for PNG glitch in 2010. Back then, it only removed the CRC32 and added it back again after the internal data was glitched.
Since then, the author has continued to rewrite the script and make improved versions of it for the purpose of using it in his own work, but he decided to make a library that adopts his know-how in 2014. The Ruby library PNGlitch came out as the result.
Every glitch image that appears in this article is made by using this library.
Appendix A explains how to use the PNGlitch library.
(The user must have a certain level of knowledge of the Ruby language in order to understand the code snippet samples.)
png = PNGlitch.open '/path/to/your/image.png'
png.glitch do |data|
data.gsub /\d/, 'x'
end
png.save '/path/to/broken/image.png'
png.close
The code above can also be written in a different way, like the one below.
PNGlitch.open('/path/to/your/image.png') do |png|
png.glitch do |data|
data.gsub /\d/, 'x'
end
png.save '/path/to/broken/image.png'
end
The glitch method handles compressed and decompressed data as a single string instance. It is handy, but on the other hand the memory usage amount can become enormous. When the memory usage is an issue, the user can write a code that uses IO instead of String like the one below.
PNGlitch.open('/path/to/your/image.png') do |png|
buf = 2 ** 18
png.glitch_as_io do |io|
until io.eof? do
d = io.read(buf)
io.pos -= d.size
io.print(d.gsub(/\d/, 'x'))
end
end
png.save '/path/to/broken/image.png'
end
PNGlitch also provides a method to manipulate each scanline.
PNGlitch.open('/path/to/your/image.png') do |png|
png.each_scanline do |scanline|
scanline.gsub! /\d/, 'x'
end
png.save '/path/to/broken/image.png'
end
The first example that uses the glitch method sometimes destroys bytes that express the filter type, so it might output a file that cannot be opened by certain viewer applications. The each_scanline method is much safer, and the memory usage is also low. It is a thorough method, but it takes more time than the glitch method.
Scanline data is made out of pixel data and the filter type value.
The user can also rewrite pixel data using Scanline#replace_data.
png.each_scanline do |scanline|
data = scanline.data
scanline.replace_data(data.gsub(/\d/, 'x'))
end
The user can also use Scanline#gsub! and do treatments like String#gsub!.
png.each_scanline do |scanline|
scanline.gsub! /\d/, 'x'
end
The user can confirm the filter type of the PNG file by running the command below. Internally, the filter types None, Sub, Up, Average and Paeth are all expressed by numeric values between 0 and 4.
puts png.filter_types
The user can also check each filter type using each_scanline.
png.each_scanline do |scanline|
puts scanline.filter_type
scanline.change_filter 3
end
The sample above has had each filter type changed to 3 (Average). change_filter properly applies the new filter type. This treatment will not cause glitches to occur because the filter is re-calculated and the PNG will be properly formatted. This also means that the resulting image will appear as the same to our eyes.
However, the difference of each filter has a large influence on the glitches.
PNGlitch.open(infile) do |png|
png.each_scanline do |scanline|
scanline.change_filter 3
end
png.glitch do |data|
data.gsub /\d/, 'x'
end
png.save outfile1
end
PNGlitch.open(infile) do |png|
png.each_scanline do |scanline|
scanline.change_filter 4
end
png.glitch do |data|
data.gsub /\d/, 'x'
end
png.save outfile2
end
The output results of the two samples above are completely different. The difference is in the filters.
The code examples that I have explained are all manipulations done to the “Filtered Data” state. When the users want to glitch “Compressed Data” in PNGlitch, they must use the glitch_after_compress method.
png.glitch_after_compress do |data|
data[rand(data.size)] = 'x'
data
end
"The PNGlitch library is released as open source.
https://github.com/ucnv/pnglitch
Appendix B includes a list of glitch variations that were not covered in the main article. This catalogue will reveal how wide the variety of PNG glitch expressions can be.
I will define 3 simple methods to destroy data.
It mentions five types of filters which are: Sub, Up, Average, Paeth, and the optimized and combined filter.
It also shows 120 patterns of combinations of if there is an alpha or not, if it is interlaced or not, and which state was glitched.
The generating script is shown at the end.
require 'pnglitch'
count = 0
infiles = %w(lena.png lena-alpha.png)
infiles.each do |file|
alpha = /alpha/ =~ file
[false, true].each do |compress|
[false, true].each do |interlace|
infile = file
if interlace
system("convert -interlace plane %s tmp.png" % infile)
infile = 'tmp.png'
end
[:optimized, :sub, :up, :average, :paeth].each do |filter|
[:replace, :transpose, :defect].each do |method|
count += 1
png = PNGlitch.open infile
png.change_all_filters filter unless filter == :optimized
options = [filter.to_s]
options << 'alpha' if alpha
options << 'interlace' if interlace
options << 'compress' if compress
options << method.to_s
outfile = "lena-%03d-%s.png" % [count, options.join('-')]
process = lambda do |data, range|
case method
when :replace
range.times do
data[rand(data.size)] = 'x'
end
data
when :transpose
x = data.size / 4
data[0, x] + data[x * 2, x] + data[x * 1, x] + data[x * 3..-1]
when :defect
(range / 5).times do
data[rand(data.size)] = ''
end
data
end
end
unless compress
png.glitch do |data|
process.call data, 50
end
else
png.glitch_after_compress do |data|
process.call data, 10
end
end
png.save outfile
png.close
end
end
end
end
end
File.unlink 'tmp.png'
A PNG scanline consists of a combination of a filter type byte and filtered pixel data. Deliberately making an incorrect combination is another technique in PNG glitching.
The image above is generated by the code below.
In PNGlitch, the method graft is prepared so that the user can attach an incorrect filter type to a scanline.
require 'pnglitch'
PNGlitch.open('png.png') do |png|
png.each_scanline do |line|
line.graft rand(5)
end
png.save "png-glitch-graft.png"
end
This technique is convenient, even for checking how different the glitching effect of each filter is. Next five images are the results that applied one particular filter type byte to every scanline, without modifying scanline data.
require 'pnglitch'
(0..4).each do |filter|
PNGlitch.open('png.png') do |png|
png.each_scanline do |line|
line.graft filter
end
png.save "png-glitch-graft-#{filter}.png"
end
end
What will happen if an incorrect filter is implemented? PNGlitch is designed to allow the user to freely change filter methods, so the user can test what happens at that state. A normal viewer application that uses a standard filter method is decoding a PNG image that is encoded by a distinctive filter method. This will perhaps generate an algorithmic glitch (I will not argue about if we should call that a glitch or not). The images below are part of such generated images.
require 'pnglitch'
PNGlitch.open('png.png') do |p|
p.each_scanline do |l|
l.register_filter_encoder do |data, prev|
data.size.times.reverse_each do |i|
x = data.getbyte(i)
v = prev ? prev.getbyte(i - 1) : 0
data.setbyte(i, (x - v) & 0xff)
end
data
end
end
p.output 'png-incorrect-filter01.png'
end
require 'pnglitch'
PNGlitch.open('png.png') do |p|
p.change_all_filters 4
p.each_scanline do |l|
l.register_filter_encoder do |data, prev|
data.size.times.reverse_each do |i|
x = data.getbyte(i)
v = prev ? prev.getbyte(i - 6) : 0
data.setbyte(i, (x - v) & 0xff)
end
data
end
end
p.output 'png-incorrect-filter02.png'
end
require 'pnglitch'
PNGlitch.open('png.png') do |png|
png.change_all_filters 4
sample_size = png.sample_size
png.each_scanline do |l|
l.register_filter_encoder do |data, prev|
data.size.times.reverse_each do |i|
x = data.getbyte i
is_a_exist = i >= sample_size
is_b_exist = !prev.nil?
a = is_a_exist ? data.getbyte(i - sample_size) : 0
b = is_b_exist ? prev.getbyte(i) : 0
c = is_a_exist && is_b_exist ? prev.getbyte(i - sample_size) : 0
p = a + b - c
pa = (p - a).abs
pb = (p - b).abs
pc = (p - c).abs
pr = pa <= pb && pa <= pc ? a : pb <= pc ? b : c
data.setbyte i, (x - pr) & 0xfe
end
data
end
end
png.output 'png-incorrect-filter03.png'
end
require 'pnglitch'
PNGlitch.open('png.png') do |p|
p.change_all_filters 2
p.each_scanline do |l|
l.register_filter_encoder do |data, prev|
data.size.times.reverse_each do |i|
x = data.getbyte(i)
v = prev ? prev.getbyte(i) : 0
data.setbyte(i, (x - v) & 0xfe)
end
data
end
end
p.output 'png-incorrect-filter04.png'
end