If you move your business within your state, you must notify:
1. The IRS.
You don't have to change any existing documents, like your employer ID number (EIN), but you must complete Form 8822-Change of Address (Part II). On the form, you can designate whether you want to change just changing your mailing address or also if you want to change the address where you receive notification of matters relating to your business income, excise, employment, and other tax matters.
2. State Secretary of State.
Notify your state's secretary of state if you have moved within your state. Each state's procedure is a little different, but you will probably have to amend your organizing documents (Articles of Incorporation for a corporation or Articles of Organization for a limited liability company). If you are a sole proprietor and you have registered your business name in your state, you may have to inform the secretary of state of this change of address.
3. State Department of Revenue.
Notify your state department of revenue (or equivalent) about your new address, so they know where to send information on sales taxes, state income taxes, and other state taxes.
3. City and County.
For licenses and permits, contact the locality and the county where you are doing business, to let them know of the change to your address.
What if You Move Out of the City/County/State?
1. City and County.
If you move out of your city, let the old city know you are gone, and get a business license with the new city. You should do the same with your county, including filing a new fictitious name/DBA statement with the new county.
If you move to a new state, you will need to register with your new state, through the state's secretary of state. You may not need to set up a new LLC in the new state, but you may be able to register as a "foreign" entity in the new state. For example, you may be ablt to register as a foreign LLC in the new state. Each state's requirements are different, so check with the new state. You must also register with the new state's Department of Revenue, to collect state sales taxes and for other taxes.
Notify the IRS using Form 8822, as above. There should be no other changes needed for the IRS notification.