This rejection is twofold.
1) I suspect that Girardi's next in line for the Yankee job when the inevitable happens for Joe Torre this October.
I can't help but believe after coming off the successful run he had with the Marlins in his one season at the helm, he didn't have his fair share of respectable offers. I know for certain Texas, Washington and the Cubs all were more than interested in him, and I'm pretty sure San Diego showed some interest as well. Yet, at the end of the day, he wound up in the Yankee Broadcast booth?
Something's more than meets the eye here. I think that after last season, Torre was told that one way or another this is his last season as Yankee manager. I don't think George had the balls to face the backlash of firing him, so he told Torre he'd give him one last year as manager and would allow him to walk away gracefully (take another job within the organization) at season's end. By giving Girardi a job in the broadcast booth, they've already got him ont he payroll and don't have to go outside the organization to hire him.
2) He knows that the Orioles are a rudderless ship, and that relationship could only end badly.
Right now, the best thing for the Orioles would be Peter Angelos selling the team to the Ripken group. As long as he's owner, the O's will never advance beyond that middle pack in the AL.