Skip to main content

Can AI Write Your alt Attributes?

Dog chasing ball

In 2016, Facebook unveiled their AI (artificial intelligence) technology, which attempts to automatically describe images to benefit people who are blind or visually impaired. While we are pretty excited about its potential, it’s important to understand where this technology might be helpful, and what its limitations are.

One of the most fundamental tasks when making a website accessible is ensuring that all of your images are also accessible to people who can’t see them. This generally involves adding alt attributes to the image tags. But what if, like Facebook, you allow your users to upload their own photos? How do you ensure that the thousands of images being uploaded each minute are properly described? Until recently, you couldn’t. But this solution is far from perfect.

To illustrate the problems with this technology, let’s look at a few examples of some randomly chosen images, along with what some AI image recognition services think they contain. You’ll notice that most of this output would need to be massaged into an appropriate format for an alt attribute, but that’s a relatively simple matter. The more important question is whether or not the information generated by these services sufficiently describes the images.

Image Human Crafted Alt Attribute Facebook Alt Attribute Amazon Output Google Output Microsoft Output
couple standing with labradoodle and holding 8 puppies couple standing with labradoodle and holding 8 puppies Image may contain: 2 people, people smiling, dog, tree, and outdoor
  • Animal
  • Canine
  • Dog
  • Pet
  • Puppy
  • Flora
  • Jar
  • Plant
  • Potted Plant
  • Pottery
  • Vase
  • Golden Retriever
  • Adorable
  • Chow
  • Blossom
  • Flower
  • Backyard
  • Outdoors
  • Yard
  • Toy
  • Tree
  • Cherry Blossom
  • Golden Retriever
  • Labrador Retriever
  • Pit Bull
  • Pug
  • French Bulldog
  • Puppy
  • Keri Barker
  • Animal
  • Retriever
  • Pet
  • Dog breed
  • Companion dog
  • Image
  • Cuteness
  • Dog
  • Grass
  • Dog
  • Outdoor
  • Person
  • Brown
  • Photo
  • Posing
  • Man
  • Standing
  • Bench
  • Front
  • Sitting
  • Holding
  • Truck
  • Black
  • Park
  • Large
  • White
  • Hot
  • Red
  • Woman
  • Field
woman wearing sunglasses and smiling in swimming pool woman wearing sunglasses and smiling in swimming pool Image may contain: 1 person, smiling, pool and outdoor
  • Pool
  • Water
  • Sport
  • Sports
  • Swimming
  • Building
  • Hotel
  • Resort
  • Swimming Pool
  • House
  • Housing
  • Villa
  • Beirut
  • Cairo
  • Yara
  • Swimming Pool
  • Image
  • Artist
  • Swimming
  • Film
  • Resort
  • Swimming
  • Recreation
  • Sabah
  • Mahmoud Hemida
  • Faten Hamama
  • Lebanon
  • Water
  • Pool
  • Building
  • Outdoor
  • Swimming
  • Table
  • Blue
  • Sitting
  • Man
  • Umbrella
  • Boat
  • Dock
  • Pond
  • Standing
  • Bird
  • Dog
  • People
  • Lake
  • Top
  • Large
  • Body
  • Woman
  • White
  • Young
  • Beach
  • Playing
  • Frisbee
  • Eating
  • Board
  • Riding
  • Court
  • Ocean
new yorker style cartoon of man being visited in the hospital. He's completely in traction and has a balloon tied to the foot of his bed. The caption says quote any improvement since I brought the balloon? end quote new yorker style cartoon of man being visited in the hospital. He’s completely in traction and has a balloon tied to the foot of his bed. The caption says quote any improvement since I brought the balloon? end quote Image may contain: drawing
  • Art
  • Drawing
  • Sketch
  • Diagram
  • Plan
  • Doodle
  • Text
  • Label
  • The New Yorker
  • Cartoon
  • Humour
  • Comics
  • Drawing
  • Image
  • Sketch
  • Magazine
  • Cartoons Magazine
  • Book
  • Canvas Print
  • Gary Larson
  • Julia Suits
  • Cartoon Cartoons
  • Farley Katz
  • None
graphite pencil sketch of a person's face graphite pencil sketch of a person’s face Image may contain: drawing
  • Art
  • Drawing
  • Sketch
  • Sketch
  • Drawing
  • Sketchbook
  • Visual Arts
  • Art
  • Pencil
  • Hatching
  • Illustration
  • Figure Drawing
  • Line Art
  • Work of Art
  • Portrait
  • Portrait Drawing
  • Cartoon
  • Nose
  • Drawing
blacktop parking spaces with the updated, forward leaning wheelchair accessible icons painted in each space blacktop parking spaces with the updated, forward leaning wheelchair accessible icons painted in each space No automatic alt text available
  • Business Card
  • Paper
  • Text
  • Asphalt
  • Tarmac
  • Disability
  • Disabled Parking Permit
  • International Symbol of Access
  • Car Park
  • Accessibility
  • Symbol
  • Wheelchair
  • Sign
  • Americans with Disabilities Act of 1990
  • Don’t DIS my ABILITY
  • Image
  • Tactile Paving
  • Independent Living
  • Services and supports for people with disabilities
  • Sticker
  • Road
  • Building
  • Tennis
  • Man
  • Court
  • Holding
  • Blue
  • Street
  • Playing
  • Sign
  • Parking
  • Game
  • City
  • Player
  • Ball
  • Standing
muscular man lifting barbell in gym muscular man lifting barbell in gym Image may contain: 1 person
  • Human
  • People
  • Person
  • Exercise
  • Fitness
  • Gym
  • Sport
  • Sports
  • Working Out
  • Building
  • Factory
  • Martini
  • Power Lifting
  • Barbell
  • Squat
  • Dumbbell
  • Strength Athletics
  • Deadlift
  • Bodybuilding
  • Bodypump
  • Exercise Machine
  • Physical Exercise
  • Human Back
  • Bar
  • Calf
  • Dorothy Parker
  • Indoor
  • Sport
  • Person
  • Standing
  • Woman
  • Holding
  • Black
  • Young
  • Man
  • Kitchen
  • Room
  • Table
  • Top
  • Large
  • Wearing
  • Red
  • Dog
  • Display
  • White
  • Luggage
  • Playing
  • Ball
  • Plane
Star Wars droid BB-8 Star Wars droid BB-8 No automatic alt text available
  • Flora
  • Food
  • Plant
  • Produce
  • Pumpkin
  • Squash
  • Vegetable
  • Inflatable
  • Figurine
  • Toy
  • Bowl
  • BB-8
  • Chewbacca
  • Jack-o-lantern
  • Star Wars
  • Pumpkin
  • Lantern
  • Droid
  • Star Wars Opening Crawl
  • Porg
  • Halloween
  • Trick-or-treating
  • Carving
  • Stencil
  • Star Wars: The Last Jedi
  • Outdoor
  • Sitting
  • Small
  • Orange
  • White
  • Table
  • Lamp
  • Top
  • Sidewalk
  • Vase
  • Flower
  • Black
  • Large
  • Green
  • Street
  • Standing
  • Yellow
  • Food
  • Dirty
  • Parked
  • Plate
row of parked school buses row of parked school buses Image may contain: outdoor
  • Bus
  • Transportation
  • Vehicle
  • School Bus
  • Cable Car
  • Streetcar
  • Tram
  • Trolley
  • Greenville County School District
  • School Bus
  • Bus
  • Elizabeth Public Schools
  • School
  • Transport
  • School District
  • School Bus Contractor
  • Public Transport
  • Field Trip
  • School Bus Yellow
  • Education
  • Seat Belt
  • Road
  • South Carolina
  • Yellow
  • Outdoor
  • Road
  • Transport
  • Bus
  • School
  • Truck
  • Parked
  • Large
  • Black
  • Lot
  • Street
  • Sitting
  • Parking
  • Side
  • City
  • Driving

As you can see from the information in the previous table, even when AI recognizes what is in an image, the amount of detail provided is far from sufficient in most cases. These algorithms also have difficulty determining what elements in the image are important, and which are simply incidental. As a result, while you can sometimes tell which objects might be in the image, it can be quite difficult to derive what the image is trying to convey.

In short, while this technology does show a great deal of promise, it should, at present, only be used in situations where it is entirely impossible to provide descriptions in any other way. Most alt attributes on the web should continue to be written by humans for the foreseeable future. On the other hand, this technology is better than nothing, though perhaps only just.

For more on what Facebook has been doing in this regard recently, visit https://www.cnet.com/news/facebook-ai-blind-photo-recognition-caption/.

← Return to the blog

Start My Free Quick Look Now

1422 W Lake Street
Suite 314
Minneapolis, MN 55408

ph: 612-440-3601

info@accessible360.com