Who knew Alcatraz—a prison so infamous it’s known simply as “The Rock”—was a garden paradise? I certainly didn’t. On a recent travel assignment in San Francisco, I decided to take the ferry to Alcatraz Island because ferry excursions on San Francisco Bay are always fun, but mostly because I’d never been to this long-time tourist attraction.

When it comes to notable federal penitentiaries I’m no scholar; a fellow blogger suggested watching the 1979 movie “Escape From Alcatraz” before I visited, but I didn’t get around to it. And while the island is an iconic part of the San Francisco urban landscape, from a geographic perspective it’s little more than a bump in the bay compared to nearby, much larger Angel Island.

I purchased my timed ticket in advance to make sure I got a spot on the first ferry, which departs Pier 33 (the only departure point for day trips to Alcatraz) at 9:10 a.m. The weather had been really nice for mid-May in San Francisco—in other words, mostly sunny—but on the morning of my tour it was gray and misty. Quite frankly it was appropriately sepulchral, given our destination.

The ferry boats are enclosed, but there are plenty of windows, and you can also stand on the front and rear decks. The ride takes about 15 minutes, and on the way a guy turned to me and commented, “It’s really not that big of a rock.” He was right; as the ferry pulled up to the dock you get a much better sense of the island’s small size. I also wasn’t expecting to see trees, but there are a couple of big eucalyptus trees on the island, and their aromatic fragrance filled the damp air.

You have the option of taking a guided tour or poking around on your own, and naturally I chose the latter. I also skipped the orientation talk at the dock, although before setting out I did hear one little factoid. The water in the bay averages a nippy 50-55 degrees and it’s a little over a mile between the island and Fisherman’s Wharf, effectively putting the kibosh on escape plans that involve swimming. Just for the record: In the 29 years that Alcatraz served as a federal penitentiary, 36 inmates tried to escape. All but five were recaptured, and those unaccounted for all participated in the same June 1962 breakout attempt depicted in the Clint Eastwood-starring “Escape From Alcatraz.”

The buildings, including a guard tower, water tower, power plant, the warden’s house and of course the cellblocks, are in various states of disrepair. Weeds sprout from cracks in the recreation yard, a dismal-looking, fenced-in concrete rectangle. The prison cells—each furnished with a toilet, tiny sink and single bunk—are depressingly small and grim. I spent about 5 minutes checking out the communal shower area and imagining, with a shudder, what it was like being cooped up in one of those cells before leaving the dank and claustrophobic atmosphere behind and stepping outside. Downtown San Francisco looked tantalizingly close from the observation deck where the island lighthouse once stood—and at the same time so very far away. Bird droppings spatter the concrete everywhere, a sort of nose-thumbing rebuke to this once-formidable outpost; after all, the birds can leave The Rock whenever they want.

Then I discovered the gardens. There are several flower beds along East Road, standing out in manicured juxtaposition to the rubble of concrete blocks and rusted metal. But it’s the gardens along the West Road that are an eye-catching surprise. The steep, rocky slope below the lighthouse is covered with Persian carpet, the common name for a dwarf perennial that forms a dense blanket of fleshy green leaves and tiny pink flowers. It was planted not only to provide a bit of beauty but to control erosion.

The Gardens of Alcatraz were created not only by the families of correctional officers but by the inmates themselves. The island’s natural environment—rocky ground, poor soil, a scarcity of fresh water and chilly, salt-laden winds—is not conducive to horticulture, but Alcatraz’s early gardeners chose hardy, sustainable plants from places like South Africa and the Mediterranean that have a climate similar to San Francisco’s. When the prison closed in 1963 the carefully tended landscape became overgrown and wild, but since 2003 a crew of volunteers has worked to restore and maintain these historic gardens.

Fuschias, geraniums, nasturtiums and bearded irises flourished along rocky walls and in flower beds next to ruined buildings. Creamy white calla lilies accented stone walls. And aeoniums were everywhere. These low-growing, saucer-shaped succulents, a common sight in urns and sidewalk gardens throughout San Francisco, come in different colors and have cool-looking leaves in the shape of a rosette.

I even discovered a plant new to me: Gordonia axillaris, more commonly known as the fried egg plant for obvious reasons. It had the added advantage of the Golden Gate Bridge as a scenic backdrop. And because there are no natural predators, Alcatraz Island is a bird sanctuary for such species as the Western gull—they were everywhere—cormorants, black-crowned night herons and snowy egrets. The egrets produce a gurgling noise that sounds nothing like a bird but something like a baby calling out the word “mama.” They were nesting and stayed out of sight, but the unusual sound attracted lots of attention.

By noon—as often happens here—the thick clouds had broken up to reveal a glorious blue sky. The sun, the plants, the constant cries of the seagulls and the distant clang of boat whistles combined to make Alcatraz seem like a really pretty and peaceful place, and I hung around until 1:30. By then The Rock was crawling with tourists and packs of noisy kids on school field trips, and I stood waiting in a long line for the next ferry. It’s another reason to get here early, because you pretty much have the place to yourself. But I thoroughly enjoyed my Alcatraz sojourn. Come for the history, but linger for the unexpected beauty.