1. 1. Working with Digital Images Dr Sgolne M. Tarte Digital.Humanities@Oxford Summer School 23rd July 2015 University of Oxford, UK
2. 2. Digital.Humanities@Oxford Summer School 23rd July 2015, University of Oxford, UK S. Tarte Introduction to the Digital Humanities: Why images? To preserve, conserve, and curate To analyze, study, and interpret To document, present, and disseminate And because they: Are portable Can be processed without damage to the pictured object Can give access to new information: Multispectral imaging: seeing beyond visible light Faces and surfaces hidden in exhibitions etc
3. 3. Digital.Humanities@Oxford Summer School 23rd July 2015, University of Oxford, UK S. Tarte Introduction to the Digital Humanities: 0 0 0 0 0 0 0 0 0 0 0 127 0 255 255 255 0 0 255 0 0 255 0 0 0 255 0 0 255 0 255 0 0 255 0 0 0 255 0 0 255 0 255 255 255 255 0 0 0 255 0 0 255 0 255 0 0 255 0 0 0 255 255 255 0 0 255 0 0 255 0 0 0 0 0 0 0 0 0 0 0 0 0 127 What are digital images? For a grey-scale image (8bit): An array of integers with values between 0 and 255 (or 256 values between 0 and 1) 0 is black 255 (1) is white Each cell in the array is a pixel, with: Coordinates (x, y) A pixel value v between 0 and 255 A pixel size defining the resolution of the image 0 0 0 0 0 0 0 0 0 0 0 127 0 255 255 255 0 0 255 0 0 255 0 0 0 255 0 0 255 0 255 0 0 255 0 0 0 255 0 0 255 0 255 255 255 255 0 0 0 255 0 0 255 0 255 0 0 255 0 0 0 255 255 255 0 0 255 0 0 255 0 0 0 0 0 0 0 0 0 0 0 0 0 127
4. 4. Digital.Humanities@Oxford Summer School 23rd July 2015, University of Oxford, UK S. Tarte Introduction to the Digital Humanities: What are digital images? For a colour image: Up to 4 arrays, or channels, storing values according to a given model of colour space Examples of colour spaces: HSL Hue Saturation Lightness RGB Red Green Blue HSV Hue Saturation Value RGBA Red Green Blue Alpha CMYK (for printing) Cyan Magenta Yellow blacK
5. 5. Digital.Humanities@Oxford Summer School 23rd July 2015, University of Oxford, UK S. Tarte Introduction to the Digital Humanities: Two models of colour spaces RGB Red value, r Green value, g Blue value, b Note: if r=g=b, then the colour is on the grey scale HSL Hue, h Saturation, s Lightness, l Notes: if s=0, then the colour is on the grey scale; if l=0, the colour is black; if l=1, the colour is white
6. 6. Digital.Humanities@Oxford Summer School 23rd July 2015, University of Oxford, UK S. Tarte Introduction to the Digital Humanities: Discrete light receptors in the retina: Cones photopic vision: 6-7 million Highly sensitive to colour (specialised red, green, blue cones) Concentrated around the fovea Bright-light vision Fine details Rods scotopic vision: 75-150 million Sensitive to low levels of illumination Large area of distribution on the retina Dim-light vision Overall picture of the field of view Elements of human visual perception
7. 7. Digital.Humanities@Oxford Summer School 23rd July 2015, University of Oxford, UK S. Tarte Introduction to the Digital Humanities: Elements of human visual perception Mach Band effect Optical illusions
8. 8. Digital.Humanities@Oxford Summer School 23rd July 2015, University of Oxford, UK S. Tarte Introduction to the Digital Humanities: Digital images and visual perception Parallel between RGB colour space and the cones of our visual system RGB appropriate for fine details detection Perception of brightness is adaptive and important in detection of changes HSLs saturation channel and grey-scale images appropriate for feature detection Visual perception is context dependent and encapsulates (implicit) expectations and knowledge Choosing how to look at images and how to process them (as well as, upstream, how to capture them!!) is interpretative
9. 9. Digital.Humanities@Oxford Summer School 23rd July 2015, University of Oxford, UK S. Tarte Introduction to the Digital Humanities: Woodgrain removal: asset or hindrance?
10. 10. Digital.Humanities@Oxford Summer School 23rd July 2015, University of Oxford, UK S. Tarte Introduction to the Digital Humanities: Digital is not neutral! Digitized versions of an artefact are digital avatars of the artefact Digital avatars: (1) Are encoded (2) Are embedded into the real (3) Influence the real Express a certain form of presence of the artefact (re- materializaton) Are contingent on the intention of the act of digitization Have an expected performative value
11. 11. Digital.Humanities@Oxford Summer School 23rd July 2015, University of Oxford, UK S. Tarte Introduction to the Digital Humanities: As for texts, images require: Provenance Who made the image? From what? How? Why? When? Processing principles [// Editorial principles] For what purpose was the image produced/modified? Was it modified/processed? If so, how and why? Processing is political [All mark-up is political]
12. 12. Digital.Humanities@Oxford Summer School 23rd July 2015, University of Oxford, UK S. Tarte Introduction to the Digital Humanities: Histogram-based processing A histogram visualizes the distribution of grey levels in the image: to each grey levels value v (bin in the histogram) corresponds the count N of pixels with this grey level value v (N is the height of the histogram bar, for the bin v) Note: All principles of processing presented hereafter will deal with 8bit grey-scale images but can be applied to colour images by applying to each channel of the adopted colour model] 0 0 0 0 0 0 0 0 0 0 0 12 7 0 25 5 25 5 25 5 0 0 25 5 0 0 25 5 0 0 0 25 5 0 0 25 5 0 25 5 0 0 25 5 0 0 0 25 5 0 0 25 5 0 25 5 25 5 25 5 25 5 0 0 0 25 5 0 0 25 5 0 25 5 0 0 25 5 0 0 0 25 5 25 5 25 5 0 0 25 5 0 0 25 5 0 0 0 0 0 0 0 0 0 0 0 0 0 12 7
13. 13. Digital.Humanities@Oxford Summer School 23rd July 2015, University of Oxford, UK S. Tarte Introduction to the Digital Humanities: Example Image size: 6048x4032 LinearcountscaleLogarithmiccountscale
14. 14. Digital.Humanities@Oxford Summer School 23rd July 2015, University of Oxford, UK S. Tarte Introduction to the Digital Humanities: Brightness and contrast adjustments Brightness: shifts the histogram towards the whites (255) to brighten; shifts the histogram to the blacks (0) to darken. The pixels are redistributed in the bins of the histogram:
15. 15. Digital.Humanities@Oxford Summer School 23rd July 2015, University of Oxford, UK S. Tarte Introduction to the Digital Humanities: Brightness and contrast adjustments Contrast: redistributes the pixel colours so that they span more grey values for more contrast, (resp. less grey values, less contrast) The pixels are redistributed in the bins of the histogram:
16. 16. Digital.Humanities@Oxford Summer School 23rd July 2015, University of Oxford, UK S. Tarte Introduction to the Digital Humanities: Image segmentation Image segmentation is the action of determining region(s) of particular interest (ROI) in an image, e.g. script, brush strokes The crucial task: translate into image/pixels terms what a region of interest is: Specific structures (usually called features in image processing terms) Areas sharing a given property, a form of similarity
17. 17. Digital.Humanities@Oxford Summer School 23rd July 2015, University of Oxford, UK S. Tarte Introduction to the Digital Humanities: Image segmentation Two main strategies to identify ROIs: Feature detection: detect features, i.e. where there are discontinuities in the grey values (like at the edges of the Mach bands) Example: Finding blobs, lines, and edges Region identification: define regions, i.e. where there is a form of continuity/similarity between pixels Example: Finding areas, patches
18. 18. Digital.Humanities@Oxford Summer School 23rd July 2015, University of Oxford, UK S. Tarte Introduction to the Digital Humanities: Feature detection Related to brightness perception Easier done after having transformed the image into so-called Fourier space (which deals with frequencies) Useful filters: Sobel filter, differential filter, Canny edge detector (edges) Laplace filter, Difference of Gaussians (blobs) Hough transform (ridges) These filters work by identifying specific behaviours of the image expressed in Fourier space, it then isolates those behaviours in Fourier space and returns the corresponding areas in image space. Other filters: High-pass: sharpening (keep the fine details) Low-pass: smoothing and blurring (keep the larger areas)
19. 19. Digital.Humanities@Oxford Summer School 23rd July 2015, University of Oxford, UK S. Tarte Introduction to the Digital Humanities: Feature detection [ generated in Gimp 2.8 SobelFilter]
20. 20. Digital.Humanities@Oxford Summer School 23rd July 2015, University of Oxford, UK S. Tarte Introduction to the Digital Humanities: Feature detection [ generated in Gimp 2.8 Despeckle + DoG 14- 12]
21. 21. Digital.Humanities@Oxford Summer School 23rd July 2015, University of Oxford, UK S. Tarte Introduction to the Digital Humanities: Region identification Related to colour/grey-level perception Thresholding Histogram-based classification of pixels into foreground/background by mapping selected values onto black or white Region growing (magic wand / fuzzy selection colour selection) Starts at a so-called seed point, defined manually Based on a similarity criterion (allowed colour variation) and (for the fuzzy selection) connectivity of the similar pixels
22. 22. Digital.Humanities@Oxford Summer School 23rd July 2015, University of Oxford, UK S. Tarte Introduction to the Digital Humanities: Thresholding
23. 23. Digital.Humanities@Oxford Summer School 23rd July 2015, University of Oxford, UK S. Tarte Introduction to the Digital Humanities: Region growing
24. 24. Digital.Humanities@Oxford Summer School 23rd July 2015, University of Oxford, UK S. Tarte Introduction to the Digital Humanities: Multiple images, getting more information Multi-spectral imaging Changing illumination conditions: Reflectance Transformation Imaging (RTI)
25. 25. Digital.Humanities@Oxford Summer School 23rd July 2015, University of Oxford, UK S. Tarte Introduction to the Digital Humanities: Multispectral imaging (MSI) MSI can typically span wavelengths in the range ~380 nm to 1100 nm Da Vincis adoration of the Magi
26. 26. Digital.Humanities@Oxford Summer School 23rd July 2015, University of Oxford, UK S. Tarte Introduction to the Digital Humanities: MSI MSI also relies on the light absorption and reflective properties of the components of the artefact being imaged The (mineral & organic) chemical components react differently to different wavelengths Possibility to isolate components
27. 27. Digital.Humanities@Oxford Summer School 23rd July 2015, University of Oxford, UK S. Tarte Introduction to the Digital Humanities: MSI
28. 28. Digital.Humanities@Oxford Summer School 23rd July 2015, University of Oxford, UK S. Tarte Introduction to the Digital Humanities: RTI: Allowing procedural mimesis Capture the physical characteristics of the artefact that power the sense-making process Rely on properties of the visual system Mimic a physical-world interpretation strategy of the experts Pitch-and-yaw motion in raking light Exaggeration of highlights and shadows Visual system extracts (interpolates) volumetric information (shadow- stereo principle) An aspect of materiality
29. 29. Digital.Humanities@Oxford Summer School 23rd July 2015, University of Oxford, UK S. Tarte Introduction to the Digital Humanities: How RTI works Multiple image capture 76 LEDs One picture per LED Create a Polynomial Texture Map (PTM; hence *.ptm files) Extract a base RGB image Based on a luminance model of light fit the changes of illumination to a quadratic surface
30. 30. Digital.Humanities@Oxford Summer School 23rd July 2015, University of Oxford, UK S. Tarte Introduction to the Digital Humanities: PTM the LRGB format principle For each pixel in a .ptm file, are stored: RGB as in other formats A red value A green value A blue value And a light channel L Does not store the 76 values of each of the captured images Instead fits a (quadratic) surface to these 76 values Only requires to store the 6 coefficients describing the surface as a function of light position Also allows to simulate light positions for which no picture was originally captured [L(lu,lv )= a0lu 2 +a1lv 2 +a2lulv +a3lu +a4lv +a5]
31. 31. Digital.Humanities@Oxford Summer School 23rd July 2015, University of Oxford, UK S. Tarte Introduction to the Digital Humanities: A Proto-Elamite tablet Louvre, Sb 02801; Source: http://cdli.ucla.edu/
32. 32. Digital.Humanities@Oxford Summer School 23rd July 2015, University of Oxford, UK S. Tarte Introduction to the Digital Humanities: Information in the difference Take advantage of the shadow stereo principle, i.e., of the motion of the shadows and highlights depending on the light position 32
33. 33. Digital.Humanities@Oxford Summer School 23rd July 2015, University of Oxford, UK S. Tarte Introduction to the Digital Humanities: How the blend modes work (Gimp / Photoshop) Layers are stacked Their order is important To each layer is associated a mode This mode defines how the current layer is combined with the layer below it Depending on the nature of the blend mode, swapping two layers (and their associated blend modes) will drastically affect the results It can be useful to have an extra empty background layer: e.g. a black layer if looking to combine all images and only see the lighter pixels or, a white layer if looking to combine all images and only see the darker pixels
34. 34. Digital.Humanities@Oxford Summer School 23rd July 2015, University of Oxford, UK S. Tarte Introduction to the Digital Humanities: Multiple images In a set of images of an object taken from the same vantage point, new information lies in how those images vary Explore the differences by using the blend modes: Difference Subtract Darken only Lighten only
35. 35. Digital.Humanities@Oxford Summer School 23rd July 2015, University of Oxford, UK S. Tarte Introduction to the Digital Humanities: Using processed images Processing images is modifying them Processing images is interpreting them Its ok to modify images if were clear about what were looking for and why we use one method or another when processing Understanding the (often black-boxed) image processing options helps justify choices and make expectations explicit the act of interpretation then becomes more sharable, reproducible, and teachable
36. 36. Digital.Humanities@Oxford Summer School 23rd July 2015, University of Oxford, UK S. Tarte Introduction to the Digital Humanities: Processing IS interpreting Its important to not mislead your audience Make your processing obvious As a process Give details of what has been done and why (expose methodology & methods) As a result Avoid smoothing stitching of images Use non-photo-realistic colours as much as possible - itll then be obvious you have done something to the images
37. 37. Digital.Humanities@Oxford Summer School 23rd July 2015, University of Oxford, UK S. Tarte Introduction to the Digital Humanities: Example of the Artemidorus papyrus A strange papyrus with Text including portions of Artemidorus geography Maps Drawings Controversies around its authenticity Its a fake: and the forger is C. Simonides(19th cent) It cant possibly be a fake, in spite of its strangeness Theory: the three lives of the papyrus: 1. early Roman period de luxe copy of the 2nd book of Artemidorus geophraphy (2nd cent BC) with maps 2. re-used in an artists workshop: verso with mythological and real animals (sketch book) 3. Verso blanks filled in with drawings of heads, hands, and feet (sketch book) [Gallazzi & Kramer, 1998]
38. 38. Digital.Humanities@Oxford Summer School 23rd July 2015, University of Oxford, UK S. Tarte Introduction to the Digital Humanities: Making the intangible tangible: P. Artemid. Virtual access to the papyrus only IR images Mirror-images through ink transfers Virtually evaluate how the papyrus was rolled Virtually compute its length Virtually reposition the fragments Re-materialization of some aspects of the papyrus [Tarte, 2012] [DAlessio, 2012] [Latour & Lowe, 2011] (in collaboration with Prof. DAlessio (KCL), and Dr Elsner (Oxford))
39. 39. Digital.Humanities@Oxford Summer School 23rd July 2015, University of Oxford, UK S. Tarte Introduction to the Digital Humanities: Example of the Artemidorus papyrus
40. 40. Digital.Humanities@Oxford Summer School 23rd July 2015, University of Oxford, UK S. Tarte Introduction to the Digital Humanities: Example of the Artemidorus papyrus Correspondences between recto and verso pictures: Measurements between original and transfer images to simulate the rolled papyrus 12.5cm at the level of V25 on the verso, which corresponds on the recto to approximately 40cm inwards of the left end of section (b+c) 13.2cm at the level of column (iv) 15.3 cm to 4cm at the level of the hands (R16, R18) at the right end of section (b+c)
41. 41. Digital.Humanities@Oxford Summer School 23rd July 2015, University of Oxford, UK S. Tarte Introduction to the Digital Humanities: Example of the Artemidorus papyrus An Indian wild beast, hybrid between wolf and dog possibly a hyena
42. 42. Digital.Humanities@Oxford Summer School 23rd July 2015, University of Oxford, UK S. Tarte Introduction to the Digital Humanities: Digital version of the hand-out: https://www.dropbox.com/s/32yks9fjxw142n6/Refere nceList-Images.docx?dl=0