When Stannis had told Ser Davos that his family is staying with him after he had told him go back to Castle Black

Smelly Sandra would have been keeping any eye on Kid Shireen for just that reason.

Well I would, but them I'm not the same kind of person as Davos. I have never possessed anything like his odd brand of blind loyalty, not that I live in a culture that encourages loyalty to lords and oath-keeping at all costs.

There are two things about taking Shireen and making a run for it, the first being that Stannis would send men after me as I fled, and I don't know how much of a danger that would have been, with Stannis's men deserting the instant they could get away from him. The other is that I would really, REALLY not want to leave a girl child at Castle Black! it wasn't quite the wretched hive of scum and villainy it was in season 1, with a lot of the bad apples deserting and getting killed at Craster's, but I wouldn't leave a little girl there if there were any alternative. And yeah, it would have been a question of leaving her there, because Ser Blind Loyalty would have felt he had to return to Stannis.

I would have taken the sprat, gone to Castle Black and stayed, and volunteered Shireen and myself as assisstants to the Maester, or substitute Maester in training.