Guards detail peek, how does it work?

What dictates a faction as friendly towards me? I have guards detail, but getting hit in the city does not have the guards help me.