Is turning any pretty colors like purple and red?
I didn't go to the emergency room for my ankle I went to a Doc-N-Box (Medical Center, Family Care etc..), paid my $20 co-pay they took x-rays, put me in a splint and wrote me a referal and called the hospital. I went straight throught admissions no waiting in the emergency room.
The Doctor at the Medical Center stated that if the fracture isn't severe. He will splint them up to stablize it, write a referal, and let you make the orthopedic appointment.
If you have good insurance stop at one those nieghborhood clinics and at least get an x-ray.
Until then keep Ice on it and keep it elevated.