Former mayoral candidates Walter Moore and Phil Jennerjahn and David Hernandez,, along with former City Council candidate Augusto Bisani are among the proponents of the effort.
Aides to Villaraigosa said he would have no comment.
The City Clerk's Office said the group has 120 days to collect nearly 250,000 signatures in order to qualify for the ballot.
Hernandez said the group, which can be reached at www.RecallCityHall.org has raised a couple of thousand of dollars and they hope to use the Internet to get supporters to circulate petitions.
Hernandez said organizers believe Villaraigosa should be recalled because of his handling of the city's budget and his proposals for rate increases at the Department of Water and Power.