Mills spent a decade as an executive at Madison Square Garden and before that worked at the NBA for 16 years. He left MSG in 2009 after the arrival of Donnie Walsh as Knicks president.
Grunwald took over when Walsh left in 2011 and constructed much of the roster that helped the Knicks win a playoff series last season for the first time since 2000. He will remain with the organization as an adviser, the Knicks said in a release Thursday.
Madison Square Garden Chairman James Dolan called Mills "a well-respected sports executive with a strong background in basketball, as well as a familiarity with NBA operations and our company."
"We look forward to his leadership and believe he is the right person to help us reach our ultimate goal of winning an NBA championship," Dolan said.
The surprising news came with the Knicks set to open training camp Tuesday coming off a 54-win season and their first Atlantic Division championship since 1994.
Grunwald tied for third in last season's voting for Executive of the Year and had made a series of moves over the summer, including signing Metta World Peace and trading for former No. 1 pick Andrea Bargnani, but now Mills will inherit the roster and try to keep strengthening it.
Mills came to MSG in 1999 and became president of MSG sports teams operations in 2001. He is perhaps best-known among fans for hiring Isiah Thomas as Knicks president in 2003. Thomas and former MSG executive Anucha Browne Sanders, who was awarded $11.5 million in 2007 after suing Thomas and the Garden for sexual harassment, both reported to Mills, who was reassigned after Thomas' removal as coach and GM in 2008.
Mills left the Garden a year later to work for Magic Johnson Enterprises and later founded Athletes & Entertainers Wealth Management Group, LLC, a company that helps sports and entertainment stars manage finances and find business opportunities.
"I want to thank Jim for this opportunity to return to the Knicks in this important position," Mills said. "It is an honor and a privilege to work for such a storied franchise and I look forward to doing whatever it takes to make the Knicks a successful franchise and a team that fans can be proud of, both now and in the future."
Mills, who played basketball at Princeton, joined the NBA in 1983 and held a number of positions, including his final role as senior vice president of basketball and player development. He had been considered a candidate to replace Billy Hunter as executive director of the NBA Players Association.