JPG file format: segments, metadata and what survives an export

A JPG file is a chain of marker segments: a two-byte marker such as FFE1, a two-byte length, then the payload. The compressed pixels start at SOS and run to EOI (FFD9). Apart from the tables the decoder needs, everything before SOS is metadata, and each standard claims its own APPn segment with an identifier string at the front. That is the jpg file format as a metadata parser sees it.

The length field is 16 bits, so no segment exceeds 65,535 bytes. Large blocks get split, and every standard splits differently.

The jpg file format, segment by segment

The APP segments of an iPhone 13 JPEG (iOS 17.1):

$ exiftool -v1 iphone13.jpg | grep -E '^JPEG (APP|SOS)'
JPEG APP0 (18 bytes):
JPEG APP1 (2844 bytes):
JPEG APP1 (992 bytes):
JPEG APP2 (86 bytes):
JPEG APP2 (30266 bytes):
JPEG APP10 (762 bytes):
JPEG APP13 (172 bytes):
JPEG SOS

The identifier at the start of each payload says what it is:

ExifTool 12.76 does not decode APP10. Decoders skip unknown segments by length.

Exif is a small TIFF file

After the six-byte Exif\0\0 header comes a full TIFF header (MM\0* in this file, big-endian), and every offset inside is counted from there. IFD0 has Make, Model and Orientation, with pointers to the ExifIFD and GPS IFD. IFD1, when present, points to an embedded JPEG through ThumbnailOffset (0x0201) and ThumbnailLength (0x0202).

That thumbnail is the classic leak. Cropping a 4032x3024 iPhone photo to 800x600 with ImageMagick 6.9.12 kept the 160x120 thumbnail of the uncropped frame, and ExifImageWidth still said 4032. exiftool -b -ThumbnailImage brings the cropped-out part back.

XMP and the 64 KB ceiling

Standard XMP sits behind http://ns.adobe.com/xap/1.0/ and a null byte. Bigger packets use Extended XMP: extra APP1 segments labeled http://ns.adobe.com/xmp/extension/, tied to the main packet by an MD5 stored in xmpNote:HasExtendedXMP. A 120 KB description written by ExifTool became one standard segment plus extensions of 65,533 and 54,985 bytes. A reader that ignores extensions gets a truncated packet, without a warning.

Raw developers cause most of the bloat: an Ubuntu wallpaper shot on a Canon EOS RP carries 42 KB of XMP, mostly darktable history.

ICC, MPF, IPTC and C2PA

The ICC block is ICC_PROFILE\0 plus a chunk number and chunk count. Profiles over 64 KB span several APP2 segments, reassembled in order.

MPF indexes images stored after the primary image's EOI. In the iPhone file, entry two is a 2016x1512 JPEG whose XMP says urn:com:apple:photo:2020:aux:hdrgainmap: the HDR gain map.

IPTC IIM lives in Photoshop resource 0x0404 inside APP13, as numbered datasets (2:80 By-line, 2:116 CopyrightNotice, 2:25 Keywords, 2:120 Caption-Abstract). Resource 0x0425 holds an MD5 of that block, used to detect IPTC edited by software that ignored the XMP.

C2PA uses APP11 JUMBF boxes, split over contiguous segments. The manifest hashes the rest of the file, so editing any other segment invalidates it. More in what C2PA Content Credentials are.

What exports keep

Photoshop's Save for Web (Legacy) has a Metadata menu that goes up to All. Export As and Quick Export only offer None or Copyright and Contact Info, so camera EXIF and GPS never survive them. Pillow 10.2 re-saving an iPhone JPEG wrote neither EXIF nor ICC unless both were passed to save(). ImageMagick copies nearly everything, stale values included.

Metadata edits do not touch the pixels

Re-encoding degrades the image; rewriting metadata does not. ExifTool rebuilds the APP segments and copies the scan data byte for byte.

$ exiftool -s -api ImageHashType=MD5 -ImageDataHash photo.jpg
ImageDataHash                   : f021390c10385c6fbe6122d27cce57c1
$ exiftool -overwrite_original -Artist="Jane Doe" -IPTC:By-line="Jane Doe" photo.jpg
    1 image files updated
$ exiftool -s -api ImageHashType=MD5 -ImageDataHash photo.jpg
ImageDataHash                   : f021390c10385c6fbe6122d27cce57c1

A Pillow save at quality 95 changed that hash, as any decode and re-encode will. Our metadata editor writes through ExifTool, so the scan data stays identical there too, and the EXIF reader shows the fields it supports before you edit.

The metadata remover runs exiftool -all=. On the iPhone 13 JPEG that deleted every APP segment, JFIF included, plus the MPF gain map: 3.15 MB became 2.79 MB. The ICC profile and Orientation go too, so a photo stored sideways with Orientation 6 displays rotated afterwards. Any full strip does this.