I've always been fascinated with this day, since I read about it in some children's horror books (by Tais Teng, Paul van Loon and other authors from the 'Griezelgenootschap'), but it's only now that I've started looking up its history.
Walpurgis Night is a traditional pagan celebration. Vikings would light bonfires to chase evil spirits away. Legend has it that the gates to hell are open on this night, and that evil spirits are able to roam the earth freely. It is also said that witches would gather on the mountain Brocken (in Germany), which is also mentioned in Goethe's 'Faust'. There are several literary works that refer to Walpurgis Night, 'Faust' is only one of them. Thomas Mann (in 'The Magic Mountain') and Edward Albee (in the play 'Who's Afraid of Virginia Woolf?'), H.P. Lovecraft and Bram Stoker all mention Walpurgis Night in their works. As I said, even some recent children's book still have scenes that occur during this night.
Nowadays, it is usually celebrated by lighting big bonfires, as the vikings did. Unfortunately, in Belgium Walpurgis Night isn't really celebrated. According to Wikipedia, in Germany, Sweden, Finland, Estonia and the Czech Republic it is a very popular festival.
Edit: while googling some more, I found out that the final scene in the Disney movie 'Fantasia' is supposed to portray Walpurgis Night!