Payroll Tax Defense — IRS & EDD
Payroll Tax Attorney
Sam Brotman defends California employers in IRS and EDD payroll tax disputes — Form 941 deficiencies, trust fund recovery penalties, EDD audit assessments, and payroll tax fraud investigations.
What Does a Payroll Tax Attorney Do?
A payroll tax attorney defends employers against IRS and state tax agency enforcement actions arising from undeposited payroll taxes, Form 941 deficiencies, trust fund recovery penalty assessments, and EDD audit findings. The defense practice covers both the civil collection side — stopping levies, negotiating installment agreements, litigating penalty abatement — and the personal liability side, where the IRS uses the Trust Fund Recovery Penalty under IRC § 6672 to pursue individual officers and owners for the company's unpaid employment taxes.
Call (619) 378-3138 to speak with a payroll tax attorney.
Payroll tax enforcement is the IRS's most aggressive collection program. The mechanics are simple: businesses are required to withhold federal income tax, Social Security, and Medicare from employee wages, match the FICA contributions, and deposit those funds to the IRS on a weekly, semiweekly, or monthly schedule. When a business falls behind — whether from a cash flow crisis, a bookkeeping failure, or a deliberate decision to defer — the IRS moves fast and the consequences are personal. The tax does not stay with the company. It follows the individuals who controlled the company's finances.
Brotman Law has represented businesses and individuals in payroll tax disputes at every stage: initial IRS contact, Revenue Officer visits, civil fraud penalty investigations, EDD Employment Tax Audits, and U.S. Tax Court litigation. This page describes our defense practice. For an explanation of how payroll taxes work mechanically, see our payroll tax guide.
What Counts as a Payroll Tax Problem
Payroll tax problems come in several distinct forms, each with a different enforcement mechanism and a different defense approach:
- Form 941 non-filing: Failure to file quarterly federal payroll tax returns. The IRS can prepare a substitute return (SFR) and assess tax plus failure-to-file penalties under IRC § 6651(a)(1).
- Failure to deposit: Timely filing without timely deposit. Failure-to-deposit penalties under IRC § 6656 range from 2% (1–5 days late) to 15% (more than 10 days after the first delinquency notice). These stack quickly.
- Trust Fund Recovery Penalty (TFRP): The personal liability assessment under IRC § 6672. This is the one that follows individuals out of bankruptcy and beyond the life of the business. See the deep-dive below.
- EDD Employment Tax Audit: California's Employment Development Department audits employer payroll records independently of the IRS. EDD has broad authority to reclassify workers, assess unreported wages, and impose penalties — including personal liability assessments similar to the federal TFRP.
- Worker classification disputes (1099 vs. W-2): When the IRS or EDD determines that workers treated as independent contractors should have been classified as employees, back employment taxes, penalties, and interest are assessed for every misclassified worker for every open year.
The Trust Fund Recovery Penalty
The Trust Fund Recovery Penalty (TFRP) is the IRS's mechanism for holding individuals personally liable for a company's undeposited payroll taxes. It covers only the "trust fund" portion of payroll taxes — the amounts withheld from employee wages (income tax withholding and the employee's share of FICA). The employer's matching FICA contribution is not part of the trust fund and is not subject to the TFRP.
The controlling statute is IRC § 6672. The IRS can assess the TFRP against any person who: (1) was a "responsible person" — someone with authority over the company's finances — and (2) "willfully" failed to collect, account for, or pay over the trust fund taxes. Both elements must be present. The IRS targets a wide group: majority shareholders, CEOs, CFOs, controllers, bookkeepers with check-signing authority, even board members who approved payroll budgets while knowing taxes were not being deposited.
The Liability Exposure
The TFRP equals 100% of the unpaid trust fund taxes. If a business has $500,000 in undeposited payroll taxes and $300,000 of that is trust fund (employee withholding), the TFRP assessment against each responsible person is $300,000 — regardless of how many people are assessed. The IRS can assess the full amount against five individuals simultaneously. It cannot collect more than 100% in total, but each person is on the hook for the entire amount until it is paid.
TFRP: How the IRS Builds the Number
The IRS reconstructs the trust fund liability quarter by quarter, using Form 941 data and payroll records. For each quarter where the trust fund was not fully deposited, the IRS computes: total employee wages × applicable withholding rate + employee FICA share. That number becomes the TFRP exposure for that quarter. In multi-year cases, the quarters compound — a business that went 8 quarters without depositing can face a TFRP that is 8 separate penalty assessments stacked together, each accruing interest separately from the date of assessment.
How We Defend TFRP Cases
There are two primary defense strategies in TFRP matters, and they are not mutually exclusive.
Disputing responsible person status. The IRS has to prove that you had the authority and the duty to see that the taxes were deposited. If your role at the company did not give you actual authority over tax deposits — if someone else controlled the bank accounts, if you were a passive investor, if you relied on a CFO or bookkeeper who withheld the relevant information from you — responsible person status can be challenged. The IRS's definition of "responsible person" is broad, but it is not unlimited.
Disputing willfulness. Willfulness under § 6672 does not require intent to defraud — it means the responsible person knew the taxes were not being deposited and allowed other creditors to be paid instead. But willfulness can be challenged in several circumstances: if the person did not know the taxes were unpaid, if they were deceived by the company's bookkeeper or CFO, or if they took affirmative steps to address the delinquency as soon as they became aware of it. Good-faith reliance on a professional who assured the person that deposits were being made can, in some cases, negate willfulness.
EDD Employment Tax Audits
The California Employment Development Department conducts its own employment tax audits independently of the IRS, and EDD audits have their own distinct features that California employers need to understand.
EDD audits focus on three issues: whether all wages are being properly reported, whether withholding is being calculated and deposited correctly, and — most significantly — whether workers classified as independent contractors should be reclassified as employees. EDD has been one of the more aggressive state agencies in contractor reclassification, and the exposure in a reclassification audit is significant: back employment taxes for every reclassified worker, for every open audit year (typically three years, extendable to eight in fraud cases), plus a 10% negligence penalty on underpaid taxes and a 25% fraud penalty if EDD believes the misclassification was intentional.
AB5 and Independent Contractor Risk
California's AB5 (effective January 1, 2020) codified the "ABC test" for independent contractor classification, making it substantially harder to classify workers as contractors in California. Under AB5, a worker is presumed to be an employee unless the hiring entity can demonstrate all three prongs: (A) the worker is free from the control and direction of the hiring entity; (B) the worker performs work outside the usual course of the hiring entity's business; and (C) the worker is customarily engaged in an independently established trade. Prong B in particular has been the source of significant dispute — businesses whose core service involves the work performed by the contractors are at the highest risk.
EDD auditors are aware of AB5 and apply the ABC test in audits of businesses with independent contractor workforces. If you have a significant 1099 workforce and have not had your classification methodology reviewed since 2020, an EDD audit is a real risk.
How California EDD Audits Differ from IRS Audits
EDD audits are administrative proceedings conducted by EDD examiners, with appeals to the California Unemployment Insurance Appeals Board (CUIAB) and ultimately to California Superior Court. They are not coordinated with the IRS by default, but EDD and the IRS do share information — an EDD reclassification finding can trigger an IRS reclassification audit for federal employment tax purposes, compounding the liability. We handle both agencies in the same matter.
The IRS Collection Machine
When payroll taxes go unpaid, the IRS collection process escalates in a predictable sequence. Understanding the sequence matters for knowing where you have leverage.
The IRS begins with automated notices — a CP501, CP503, CP504 (Notice of Intent to Levy). At the CP504 stage, the IRS has the legal authority to levy your bank accounts and receivables. A Revenue Officer is typically assigned for larger balances or repeat delinquencies. Revenue Officers have broad authority: they can issue levies, file tax liens (Notice of Federal Tax Lien, NFTL), and, in cases involving deliberate non-payment, refer the matter to IRS Criminal Investigation.
Tax liens under IRC § 6321 attach to all property and rights to property. Once a lien is filed, it becomes public record and can affect your ability to sell property, secure financing, or maintain business relationships. Payroll tax liens are particularly damaging because they are often filed on both the company and on individual responsible persons simultaneously — meaning the lien follows you personally even if the company closes.
Criminal referral happens in cases where the IRS can demonstrate intentional evasion — not just failure to pay, but affirmative conduct to conceal the liability or to pocket withheld funds. For most businesses with a cash flow problem, criminal exposure is not the immediate concern, but it needs to be evaluated early in any representation.
Penalty Abatement
Payroll tax penalties — failure-to-file, failure-to-deposit, and in some cases the trust fund recovery penalty itself — can be reduced or eliminated through formal abatement requests.
First-Time Abatement (FTA) is available under IRS Policy Statement 20-1 for taxpayers with a clean compliance history for the prior three years. FTA applies to failure-to-file and failure-to-pay penalties — it does not apply to the TFRP directly, but it can reduce the underlying § 6656 failure-to-deposit penalty that contributes to the total balance. FTA is frequently overlooked and is one of the most reliable abatement mechanisms when the client qualifies.
Reasonable cause abatement under IRC § 6724 and the general reasonable cause standard is available when the taxpayer can demonstrate that the failure to file or deposit resulted from circumstances beyond their control — a serious illness, a natural disaster, the death or incapacity of a key employee responsible for payroll, or reliance on professional advice that turned out to be incorrect. The IRS's standards for what constitutes reasonable cause are specific and the supporting documentation needs to be thorough.
Representative Matters
Each matter is different, and past results do not guarantee any particular outcome. The following examples are representative of the types of payroll tax cases we handle.
A technology startup founder was assessed a Trust Fund Recovery Penalty of approximately $800,000 covering six quarters of undeposited FICA taxes. We challenged responsible person status for two of those quarters, demonstrating that the founder had delegated full financial control to a CFO during those periods and that the CFO had withheld information about the delinquency. After a TFRP appeals conference, the IRS conceded two quarters, reducing the assessment to approximately $210,000, which was resolved through an installment agreement.
A California restaurant group with delivery operations classified its delivery drivers as independent contractors and received an EDD audit covering three years of payroll records. EDD proposed reclassifying all delivery drivers as employees and assessed back unemployment insurance taxes, SDI withholding, and penalties. We demonstrated that a subset of drivers met the ABC test prong B under a specific operational analysis and negotiated a classification settlement that reclassified approximately 40% of the proposed assessment, with the remainder resolved through an installment arrangement.
A construction company had not filed Form 941 for five quarters. The IRS had prepared substitute returns, assessed tax plus failure-to-file and failure-to-deposit penalties, and assigned a Revenue Officer. We prepared and filed the five delinquent returns (which reduced the tax below the IRS's SFR estimates), requested first-time abatement of the failure-to-deposit penalties for the first year, and negotiated a 72-month installment agreement. The Revenue Officer deferred lien filing based on the agreement and the company's compliance going forward.
Related Services
About Sam Brotman
Sam Brotman is a tax attorney with over 15 years of experience representing businesses and individuals in IRS and California tax controversies. He holds a J.D., an LL.M. in Taxation, and an MBA, and is admitted to practice before the U.S. Tax Court. His payroll tax defense practice covers the full range of federal and California employment tax enforcement — from initial Revenue Officer contact through Tax Court litigation and, where criminal exposure exists, coordination with his criminal tax defense practice.
Payroll tax problems are time-sensitive. Revenue Officer visits, bank levies, and TFRP assessments all have response windows that, once missed, are difficult to reopen. If you have received any IRS or EDD notice related to payroll taxes, schedule a consultation to understand your options and the timeline you are working with.
Frequently Asked Questions About Payroll Tax Defense
What is a payroll tax attorney?
How is the trust fund recovery penalty calculated?
Can I be personally liable for my company's payroll taxes?
What is an EDD Employment Tax Audit?
What happens if I don't pay payroll taxes?
Payroll Tax Defense — Federal and California
Know Where You Stand
Payroll tax problems move fast once the IRS assigns a Revenue Officer. We can assess your exposure — TFRP, civil penalties, potential lien filing — and lay out your options clearly. No urgency theater. Just a straight answer about what you are facing.
- Free 15-minute intro call — we will tell you exactly what your situation looks like
- IRS and EDD defense under one roof
- Completely confidential — attorney-client privilege from the first call