Just because he controls it doesnt mean it turns into a shadow card. There's no such thing as "shadow sites." They're all just sites, and sites on the table are always active no matter who played them, no matter whether they're on the adventure path or in a support area.