It is a federal crime not to file tax returns for tax periods in which taxes were due, in most cases. In most cases, the IRS will initiate criminal actions against you only if additional factors warrant such criminal tax prosecution. The most important step is to make proper arrangements through an experienced tax attorney and get into compliance with the IRS before they decide to come after you.