Do I have to believe in AI doom to join?

No — the opposite. A good AI safety community needs people saying “that argument doesn't work” and “where is the evidence?”. Sceptics are especially welcome: the group's job is getting better at figuring out what's true, including discovering when safety arguments are wrong.