the manual says under "active" that sites in any player's support area are still in play.
During your turn, only these cards in play are active:
• sites on the adventure path,
• sites in any player's support area,
• your Free Peoples cards,
• your copy of The One Ring, and
• your opponents' Shadow cards.
and under "spot" it defines what it means to spot something:
The word spot sets up a requirement for playing a card or using a special ability in conjunction with a noun such as, "To play, spot an Elf." This is equivalent to, "An Elf must be in play and active for you to play this card."
so, seeing as the site is in play and active, I would imagine you could, even if it was that player's fellowship phase.