The hit stunned the Yankee Stadium crowd, which anticipated a series-ending win in any of the Yankees' several chances at a last at-bat.
Baltimore used six relievers to shut out the Yankees in the final 7 1/3 innings, with Jim Johnson earning his second save of the postseason after blowing one Wednesday night in Game 3.
"Fortunately, or unfortunately, we've got some experience at this," Orioles manager Buck Showalter said of the extra-inning drama. "The pitching on both sides was outstanding."
Pedro Strop (1-0), who had a 6.35 ERA this season against the Yankees, pitched two scoreless innings for the win.
After splitting 22 regular-season and postseason games combined, the teams will face off again Friday to finally declare a winner. Yankees ace CC Sabathia will take on Jason Hammel.
"This is time to go," Sabathia said after Game 4. "This is what you play for. That's a good team over there, we've been battling them all year; hopefully I can go out, have a good performance and give us a chance to win."
It was the second straight game that lasted at least 12 innings for the teams that had been so tightly knotted in the standings through most of the second half of the regular season.
Game 4 ended with the Orioles, in the playoffs for the first time since 1997, still alive with a shot to upend the Yankees' roster full of All-Stars.
"I don't take for granted, at any time, what these guys have accomplished," Showalter said. "They know that. I have such tremendous respect for our guys and the level they choose to compete at. They look at tonight and tomorrow as an opportunity to shine and separate themselves."
The Orioles got a fifth-inning homer from Nate McLouth to take a 1-0 lead, the Yankees responded with a run in the sixth on an RBI groundout by Robinson Cano, and then the bullpens shut the offenses down.
But Machado, the Orioles' heralded rookie who homered in Game 3, led off the 13th with a double off David Phelps (0-1). One out later, Hardy doubled over left fielder Ichiro Suzuki's head, and the Orioles finally had another lead.
The Yankees' best chance came in the bottom of the eighth when Ichiro and Mark Teixeira led off with singles and were moved to third by a groundout by Cano.
Orioles reliever Darren O'Day, who pitched a season-high 2 2/3 scoreless innings, struck out slumping slugger Alex Rodriguez and got Nick Swisher on a fly to right to end the threat.
"I think it's been unusual to have so many passes of the baton as we've had this year without dropping it," Showalter said of his bullpen's effort.
For the second straight game, Yankees manager Joe Girardi pinch-hit for Rodriguez in what could have been the team's last at-bat, this time in the 13th. Raul Ibanez had been used earlier Thursday, and he did not come through as he did with the game-tying and game-winning homers in Game 3. Instead, Eric Chavez made the last out against Johnson. Girardi did not commit to starting Rodriguez in the deciding game, saying he would think about his lineup overnight.
Joba Chamberlain pitched a perfect 11th for the Yankees, but he left after getting struck by a broken bat as Matt Wieters singled to start the 12th inning. The ball landed in left as the bat struck Chamberlain's throwing elbow.
After the game, the Yankees announced that Chamberlain had a bruised elbow, but that X-rays were negative.
Phelps, a rookie, replaced Chamberlain to make his postseason debut. He retired the next three batters but faltered the next inning.
Much earlier, Phil Hughes and Joe Saunders continued the clubs' trend of pitchers' duels, each wiggling out of early jams.
Saunders threw 5 2/3 innings of one-run ball, while Hughes allowed just one run in 6 2/3 innings.
After a season full of back-and-forth, the teams will finally move on from each other Friday.
"Well, it's kind of what we've been used to the whole year," Girardi said. "It's kind of been a grind the whole year. It's been a fight to stay ahead of this club the whole year, and it's kind of fitting."
NOTES: Girardi's father, Jerry, died Saturday at the age of 81, but the Yankees' manager did not want to inform his players or make the news public, as he said, "I knew talking about it would make it probably even harder." So, when he first found out, as he rode the team bus, he "had tears in my eyes on the bus, so I put some sunglasses on." After the Peoria (Ill.) Journal Star published the elder Girardi's obituary Thursday, the Yankees made the news public. ... Yankees shortstop Derek Jeter's streak of 155 consecutive postseason games at shortstop ended with him starting at DH due to a bruised bone on his left foot. Jayson Nix started in his place at shortstop. They each had two hits. Girardi said postgame that he hoped Jeter could play shortstop Friday, but he was unsure. ... Rodriguez, who entered the night 1-for-12 with seven strikeouts in the series, was dropped to fifth in the order from third. He went 1-for-4 with a walk and two strikeouts before being lifted for Chavez.