Transforming OCR and other annotations

Turn text labels and traced features on a historical map into GeoJSON you can display on a modern map or analyze in a GIS. Allmaps uses the image’s Georeference Annotation to transform their shapes from pixels to longitude and latitude. Your code keeps the text and other metadata as properties of the resulting features.

This guide follows two examples: words detected by optical character recognition (OCR) around a San Francisco brewery, and buildings traced with Annotorious on a factory map in Massachusetts. Both use the same workflow: transform the geometry and retain the annotation’s contents.

A San Francisco insurance map of Enterprise Brewing Co., with thirteen OCR text polygons outlined in blue around words including BREWING, FERMENTING, LAGER, and BEER. — Thirteen OCR polygons around words including “BREWING,” “FERMENTING,” “LAGER,” and “BEER.”
San Francisco Sanborn Insurance Map, volume 5, pages 515–516
(1905), David Rumsey Map Collection.

The crop shows Enterprise Brewing Co., with labels for fermenting rooms, beer cellars, water tanks, and ice machinery. Each highlighted word has been correctly recognized by OCR and has its own polygon. These polygons locate the printed text, rather than tracing the brewery buildings.

Inside an OCR annotation

Each detected word is stored as a W3C Web Annotation. Its body contains the recognized text, while its target identifies the IIIF source image and an SVG polygon in full-resolution image pixels. Here is the complete annotation for FERMENTING:

{
  "@context": "http://www.w3.org/ns/anno.jsonld",
  "type": "Annotation",

  "body": {
    "type": "TextualBody",
    "value": "FERMENTING"
  },
  "target": {
    "type": "SpecificResource",

    "source": {
      "id": "https://www.davidrumsey.com/luna/servlet/iiif/RUMSEY~8~1~214142~5501497",
      "type": "ImageService2"
    },

    "selector": {
      "type": "SvgSelector",
      "value": "<svg><polygon points=\"2238,2595 2242,2574 2246,2548 2249,2523 2252,2498 2254,2470 2255,2446 2256,2420 2284,2424 2284,2448 2282,2475 2280,2502 2278,2529 2276,2555 2274,2580 2271,2606\" /></svg>"
    }
  }
}

A separate Georeference Annotation supplies the control points and transformation for the same image. Together, these two annotations let you transform the word’s polygon to geographic coordinates and retain its text as a GeoJSON property.

Transform the polygons

The following JavaScript loads both inputs, creates one transformer for the map, and applies the SVG conversion to each OCR selector. Each annotation in this example contains one polygon, which becomes one GeoJSON feature with the text and source image ID in its properties.

import { parseAnnotation } from '@allmaps/annotation'
import { GcpTransformer } from '@allmaps/transform'

const annotation = await fetch(
  'https://annotations.allmaps.org/maps/ad38324ab9d18a27@7eef0e705892f244'
).then((response) => response.json())
const labels = await fetch(
  'https://docs.allmaps.org/examples/ocr/san-francisco.ocr.json'
).then((response) => response.json())

const [georeferencedMap] = parseAnnotation(annotation)
const transformer = GcpTransformer.fromGeoreferencedMap(georeferencedMap)

const features = labels.map(({ body, target }) => {
  const [feature] = GcpTransformer.transformSvgStringToGeojsonFeatureCollection(
    transformer,
    target.selector.value
  ).features

  feature.properties = { text: body.value, imageId: target.source.id }
  return feature
})

const geojson = { type: 'FeatureCollection', features }

console.log(JSON.stringify(geojson, null, 2))

View the transformed labels

Here are the thirteen transformed text polygons on a modern map of San Francisco. Pan and zoom to inspect their positions:

Loading the map preview…

Download the GeoJSON to open it in QGIS or another mapping tool. The OCR data comes from Machines Reading Maps. Text on Maps is a visualization of this data, where you can explore and search for labels across the collection.

From Annotorious to GeoJSON

The same conversion works for manually drawn features. This example uses three buildings traced with Annotorious on a map of the Glendale Elastic Fabrics Company in Easthampton, Massachusetts, from the American Textile History Museum collection on Digital Commonwealth. Here the polygons outline buildings, rather than printed text:

Loading coordinate example…

Pan or zoom either view to explore the buildings; the other view follows. Both views show the transcribed text from each building annotation.

The annotation’s body is an array of text entries, and its id identifies the drawing. A separate Georeference Annotation supplies the image’s control points and transformation. Here is the complete Annotorious annotation for DYE & BLEACH HO.:

{
  "type": "Annotation",

  "body": [
    {
      "type": "TextualBody",
      "value": "DYE & BLEACH HO.",
      "purpose": "commenting"
    }
  ],
  "target": {

    "source": "https://iiif.digitalcommonwealth.org/iiif/2/commonwealth:2r36vc46x",

    "selector": {
      "type": "SvgSelector",
      "value": "<svg><polygon points=\"2413.6,2134.6 2609.5,2105.8 2700.9,2708.8 2495.3,2741.3\"></polygon></svg>"
    }
  },
  "@context": "http://www.w3.org/ns/anno.jsonld",
  "id": "#5538dd39-135a-431e-a4e4-d6e5195ac571"
}

The following code transforms each outline and retains the drawing’s text and annotation ID as GeoJSON properties:

import { parseAnnotation } from '@allmaps/annotation'
import { GcpTransformer } from '@allmaps/transform'

const annotation = await fetch(
  'https://annotations.allmaps.org/maps/654b26fa1850e5fc@26f09f0bcf3066b5'
).then((response) => response.json())
const drawings = await fetch(
  'https://docs.allmaps.org/examples/annotorious/buildings.annotations.json'
).then((response) => response.json())

const [georeferencedMap] = parseAnnotation(annotation)
const transformer = GcpTransformer.fromGeoreferencedMap(georeferencedMap)

const features = drawings.map((drawing) => {
  const [feature] = GcpTransformer.transformSvgStringToGeojsonFeatureCollection(
    transformer,
    drawing.target.selector.value
  ).features

  feature.properties = {
    annotationId: drawing.id,
    text: drawing.body.map((body) => body.value).join('\n')
  }
  return feature
})

const geojson = { type: 'FeatureCollection', features }

console.log(JSON.stringify(geojson, null, 2))