Executive Summary – 3D Photo Imaging for Condition Detection, Monitoring and Documentation

This project studied whether existing three-dimensional (3D) digital photo imaging technologies can be easily adopted by historic sites and collections for documenting and monitoring the condition of buildings and objects.  It also aimed to determine whether workflows could be established that would result in consistent and scientifically reliable 3D condition data capture.

The Problem: For over a century, conservators have relied on 2-dimensional (2D) photography to document and monitor the condition of heritage sites and objects.  When a sudden and shocking visual indicator appeared (e.g., a sudden crack in a surface), conservators took notice of what had actually been thousands of slow, tiny, incremental condition changes (e.g., tiny fissures and failures in slowly aging materials).  Conservators then photographed and monitored the area to try to determine how rapidly the deterioration was advancing.  Two factors limited our ability to detect and monitor deterioration: our ability detect slow and small “micro” changes, and our ability to compare what humans see using 3D stereo vision and what we recorded in 2D photographs.

Digital 3D images or surrogates allow us to see and measure 3D volumes and contours and they allow us to use a computer to detect and highlight very small and slowly occurring changes.  Detailed 3D models can be made using laser scanning and 2D digital photography (Photogrammetry, Reflectance Transformation Imaging –RTI- and Structured light). Each technology has condition-imaging strengths and weaknesses. In each, the finer the resolution – that is, the smaller the area captured by either a laser point reflection or a digital camera sensor pixel, the larger the data set for an entire object will be. In practice, the larger the object you want to image – an entire historic house or landscape versus a single door or wall, for example – the lower your resolution or detail will need to be or the more expensive and customized your assembly computer will have to be. Laser scanning equipment uses proprietary laser projectors, collection cameras, software and digital file formats. The files are huge – tens to hundreds of gigabytes each, and require very specialized computing hardware and software packages.

The Technological Solution: This study found that two 3D digital photographic technologies –Highlight RTI and Photogrammetry – promise to change the condition-detecting and monitoring paradigm in important and beneficial ways:

  • 3D photo images allow us to record and study surfaces and volumes in ways that mimic the stereo view of a real-time human examiner.
  • Because specular, color and texture data is digital, a viewer can selectively highlight and study details by removing the color data, changing the virtual light source and reflectivity or colorize contour levels.
  • Photography can be done by anyone with a basic knowledge of digital camera operation and a lap-top computer. Laser scanning engineers are not required.
  • Photography uses an ordinary, consumer-professional grade, digital camera. Expensive and complex laser scanning equipment is not required.
  • Capture photographs are the same, open-format 2D photos you have always taken –RAW, TIFF and JPEG. The capture data is not a proprietary laser-scan file format that requires special, brand-name software to use and view.
  • The user owns their own images and controls their naming, archiving, meta-data and digital management protocols.
  • Assembled 3D, computational meshes, solids, and RTI images and their viewers are open-source software – there are no hidden steps and every transformation of the data is documented and visible.
  • Because the capture photos are regular, open-format digital images, 3D images can be assembled with whatever future enhanced versions of 3D assembly software happen to come along. You are not dependent upon the life of one assembly software or manufacturer.
  • Capture photography can use the best digital camera the heritage site can afford. While 21 megapixel, full frame CMOS sensors with prime lenses and RAW capture capability give the highest resolution, less expensive APS-C sensor cameras that have manual capture modes (allowing the user to fix the lens aperture and focus) give reliable 3D condition documentation.

We created an interactive, web-platform blog site that documented our evolving working methodologies, equipment and resulting images. We specifically targeted the historic preservation field and publicized the project blog, videos and workflows using historic preservation groups on Linked-In, Facebook and Twitter social media outlets. While we anticipated somewhere near 3000 views of the website content during the course of the project, we were astonished by how rapidly an international audience, heritage preservation audience for the project was developed:

9000 views were from the USA and Caribbean.

4000 views were from Europe

700 views were from Central and South America.

500 views were from Asia (China does not report by Chinese government regulation)

400 views from Australia and New Zealand

325 views were from Canada

200 views were from the Middle East

70 views were from India

35 views were from Africa

15,230 views, total

Advertisements

Does RTI give repeatable and reliable normals of objects taken at different times and positions to facilitate detection of changes?

On the Linked-In discussion group Cultural Heritage Conservation Science. Research and practice’s discussion on 3-D digital imaging and photogrammetry for scientific documentation of heritage sites and collections http://linkd.in/RZMpFj , Greg Bearman wrote the following question:

“Does RTI give repeatable and quantitative set of normals good enough for looking for change? If I take an RTI set, rotate the object, let it warp a bit (flexible substrate), what do I get the second time? How do I align the datasets for comparison?

what is the system uncertainty? ie if I just take repeated images of the same object without moving anything, how well does the RTI data line up. Second, suppose I take something with some topography but is totally inflexible and cannot distort(make up test object here!) and I do repeated RTI on it in different orientations? Can I make the data all the same? If you are going to use an imaging method to determine changes in an object, the first thing to do is understand what is in inherent noise and uncertainty in the measuring system. It could be some combination of software, camera or inherent issues with the method itself”

I wrote back: “Hey Greg – tried sending response earlier last week but I do not see it!? Sorry. I’m on vacation until the 22nd – trying to recover and recharge. It is going well but I wanted to jot down my initial thoughts. One of my interns – Greg Williamson – is working on aberration recognition software that can recognize and highlight changes in condition captured by different H-RTI computational image assemblies – obviously taken at different times, but also with different equipment and with randomly different highlight flash positions. It seems, initally, that normal reflection is normal reflection, regardless of object or flash position and that the software correctly interpolates 3D positions of surface characteristics regardless of the precise position of the flash, because it is accustomed to calculating the highlights both the capture points and everywhere in between! Likewise, we have had promise with photogrammetry when the resolution of the images used to create the mesh and solids are similar. What may turn out to be key is a calibration set that will allow correction of the various lens distortions that would naturally come from different lenses. I know Mark Mudge at Cultural Heritage Imaging has suggested that we begin taking a calibration set before RTI capture, as we had before Photogrammetry. He may be working on incorporating a calibration correction into the highlight RTI Builder that CHI has made available. I’m sending this discussion along to the CHI forum at http://forums.cultur…ageimaging.org/ to see what others might have to add. When I return to work, I’ll ask Greg to give this some additional thought”

Whadaya think, Greg?

Digital Photographic 3D Imaging for Preservation: What’s the Buzz?

Why 3D imaging for conservation and preservation documentation?

It would hardly be worth the effort of learning and building skills in 3D, digital camera image capture and processing if 3D images didn’t offer conservators and preservation professionals better and more actionable information than 2D film, digital and IR/UV imaging.  Most people know 3D imaging either from the gaming and motion picture special effects industries or from splashy, well-funded 3D laser scanning projects of high profile art objects or heritage sites. But using 3D, digital photographic images to document and monitor conditions? This is a slightly more obscure use of imaging technology and one that, it turns out, is far more practical and adoptable.

What do you have in digital, 3D photographic documentation that you do not have with 2D documentation?

  • In addition to all the rich color data of a digital photographic 2D images, 3D, point-in-time surrogates contain quantifiable contours, volumes, textures, forms and transitions from one plane or material to another that can be continuously and seamlessly viewed and measured, from any vantage point around the object.
  • These digital 3D surrogates can be saved, recalled and enhanced by future software improvements.
  • The ability of 3D digital surrogates to locate, detail and highlight deterioration, damage and condition changes are legion. The obvious major advantage is that you can capture images using any resolution and focal distance that can be used for 2D digital photographs but you can view the damaged or undamaged surfaces and volumes in ways that truly replicate the stereo view of a real-time human examiner.
  • You can also view the damaged or undamaged surfaces and volumes in ways that ENHANCE the stereo views of a real-time human examiner.  Unlike looking at the actual surface or object, a viewer can selectively view the details of the object by removing the color data, changing the virtual light source, changing the virtual reflectivity of the surface features or colorizing volumetric levels.
  • Subtle changes that human conservators are trained to look-for and detect can be observed, captured, quantified and compared in ways that are far more revealing than 2D digital images.
  • 3D digital photographic surrogates, taken over time, reveal greater detail about the rate and extent of change to a feature – tiny soap micro-protrusions in an oil paint brushstroke, for example, or the slow, volumetric sag of a 20 meter earthen adobe wall.

Conservators always need to answer and document several key questions about anything they are trying to preserve:

  • Where is the damage or deterioration located?
  • What is the nature, size, extent and apparent character of the damage compared to the surrounding, undamaged areas?
  • What properties of the undamaged materials or structures have been lost or diminished and what degree of recovery is required to arrest deterioration and impart stability and functionality?
  • Are the conditions actively changing or deteriorating and at what rate?
  • What are the causes or precipitating events that result in, or accelerate deterioration and damage?
  • Do treatment strategies arrest, slow or accelerate the rate of deterioration?

3D, digital photographic surrogates greatly enhance our ability to identify, document and monitor the answers to these questions in ways that 2D photographic images cannot.

In this 8-week project we wanted to determine:

  • If off-the-shelf, high-end, consumer-grade digital cameras, open-format digital photographs, combined with a consumer-grade lap-top computer could be used to capture and assemble detailed, data-rich 3D images.
  • If three 3D imaging capture and processing techniques – highlight reflectance transformation imaging (RTI), photogrammetry and structured light imaging – were mature enough to be used, right now, in the summer of 2012, to capture accurate, detailed and digitally-rich condition information for works of art, historic objects and heritage architectural sites and features.
  • If two graduate students and two collections technicians with no prior experience in 3D imaging of any kind could become fully conversant and self reliant in capturing and assembling 3D images in only 8 weeks, under the guidance of a conservator and 3D imaging engineers.
  • If the capture and processing metadata – the digital capture conditions and digital pathways and transformations leading to the assembly of condition-detail-rich 3D images – could be completely open-source and open-format, with no proprietary file formats or data pathways. In this way, scientifically valid digital lab notebooks can be kept and evaluated for their validity, replicativity and value.  Further, with no proprietary files or computational pathways, all steps and all images belong to and reside with the public trust agency of the resource, rather than a private or commercial entity with no legal, public trust fiduciary requirements and restrictions.
  • If the digital camera images could be captured and formatted archivally, using ISO digital standards (Digital Negative or DNG) so that the images could always be used to assemble 3D digital surrogates far into the future, regardless of future improvements or changes in digital cameras, 3D assembly and editing software or computer operating systems and file formats.
  •  If the digital photographic, 3D files could be computationally compared by computer software so that small, slow, incremental changes in condition, often missed by museum and heritage site professionals, could be recognized by computer software and highlighted so that conservators could make better assessments about the active and unstable nature of damage and deterioration.