Despite more than four decades producing buttery, delicate French-style pastries in his family’s San Francisco bakery empire, the one meal that still sits prominently in Andrew Ly’s mind is a plain cup of noodles given to him in a Malaysian refugee camp in 1979 after making a harrowing escape from his native Vietnam. 

“That instant noodles was the best dinner in my lifetime,” Ly said, sitting in a conference room of his family’s Sugar Bowl Bakery factory in Hayward. “As of today, I still remember that.”

It was a bit of a harbinger, as food — namely baked goods from his family’s Sugar Bowl Bakery business — would define his life in a new world. 

Since then, that fateful cup of noodles, the fortunes of Andrew Ly and the other four Ly brothers have risen astronomically since they arrived penniless in San Francisco that year, escaping the turmoil in Vietnam in the aftermath of the Vietnam War. 

The Ly family story fits squarely in that chapter known as the American dream. 

“You come here, you work hard, you stay humble, you are kind, you keep improving,” Ly said. “Then you can make it in America.” 

Several years after arriving in San Francisco, the newly arrived Ly family worked odd jobs as seamstresses, maids, janitors. When a small coffee shop called Sugar Bowl Bakery in San Francisco’s Outer Richmond came up for sale, the family pooled $40,000 and bought it. They worked the new cafe with eyes on the future. 

“My dad reminded me back in the 80s, 'Andrew, whatever we do, don’t leave anyone behind,'" Ly recalled.

The five brothers and a sister began to buy up more coffee shops, rebranding them with the Sugar Bowl Bakery name. Each of the five families got one. The different neighborhood chefs who came to work in the businesses brought their recipes, and the shops became known for their baked goods. Ly said many of the pastries, like buttery madeleine, the flaky palmier, were recipes adopted by his family during France’s colonization of Vietnam. 

Andrew Ly considered himself the more adventurous of the brothers, looking to grow the company exponentially. The U.S. was the land of opportunity. He wanted a slice of it. 

“I was the one who was very instrumental and loved to expand,” Ly said. “Always loved to build something in America, always want to make it big.”

The expanding Sugar Bowl Bakery business began to sell its delectable wares to hotels and restaurants. They bought a building and turned it into a central commissary, supplying the other shops and the other businesses. By 2008, the burgeoning bakery empire was earning $50 million with 400 employees. Then the recession came along and the cascading economy took the businesses with it. 

“I decided to sell everything in San Francisco,” Ly said. "And then the retail locations, we couldn’t make it, so we closed it up.” 

But just as the Ly family survived the ruin of their own country only to remake itself in the U.S., the family remade the business in Hayward, establishing a factory to focus solely on their baked goods. They paired down their offerings from 750 products to just three: madeleine, palmier and brownie bites. The company now has factories in Hayward and Georgia, and sells to food giants like Costco, Safeway and Sam’s Club. 

“There’s a little saying, 'The harder you work, the luckier you get,,” said Mark Ly, Andrew Ly’s nephew, who also escaped Vietnam and is one of four nephews who now work in the company. “So we had a lot of luck.” 

The Sugar Bowl Bakery logo was a common sight around San Francisco. Deep within its seemingly simplistic design is the history of the Ly family itself. The pink lettering represents the pink pastry boxes the family once sold its products from. The brown lettering represents their Asian heritage. The top logo of a chef’s hat with the number five, represents the five brothers, who are mostly retired but still involved in the company. 

“I believe that America is a great country that gave the opportunity to all of us, regardless where you come from, who you are," Andrew Ly said.