Is the purpose mainly just to prevent spam / completely unqualified candidates? Its rather trivial to do the post for anyone with dev experience so I can't imagine its much use to actually gauge their skills
That's what we thought too - this was the definite first pass, and we always intended to have a challenge/response type question for extra credit.
But we didn't need it - it turns out the kind of people that are intrigued enough to apply, and have enough ability to install a few libraries are likely great people.
I thought employers generally gave FizzBuzz during an in-person interview. Surely a 5 minute phone call from a halfway technical interviewer could provide a way better filter than FizzBuzz, and avoid wasted time by having woefully unqualified people come into the office.
There are lots of filters, if I ever end up in a hiring role I intend to make use of several. Someone may be able to talk the jargon well enough during a 5 minute phone call but I think they should always be asked to write code at the in-person interview stage, and if they can't do FizzBuzz when they're being hired to program it doesn't matter what magic they weaved during the call.