Calloway was born on December 25, 1907 to a middle-class family in Rochester, New York, and lived there until 1918, on Sycamore Street. He was later raised in Baltimore, Maryland. His father, Cabell Calloway II, was a lawyer, and his mother, Martha Eulalia Reed, was a teacher and church organist. His parents recognized their son's musical talent, and he began private voice lessons in 1922. He continued to study music and voice throughout his formal schooling.