icon/x Created with Sketch.

Splunk Cookie Policy

We use our own and third-party cookies to provide you with a great online experience. We also use these cookies to improve our products and services, support our marketing campaigns, and advertise to you on our website and other websites. Some cookies may continue to collect information after you have left our website. Learn more (including how to update your settings) here.
Accept Cookie Policy

We are working on something new...

A Fresh New Splunkbase
We are designing a New Splunkbase to improve search and discoverability of apps. Check out our new and improved features like Categories and Collections. New Splunkbase is currently in preview mode, as it is under active development. We welcome you to navigate New Splunkbase and give us feedback.

Registration for .conf24 is open! Join us June 11-14 in Las Vegas

Register now

Accept License Agreements

This app is provided by a third party and your right to use the app is in accordance with the license provided by that third-party licensor. Splunk is not responsible for any third-party apps and does not provide any warranty or support. If you have any questions, complaints or claims with respect to this app, please contact the licensor directly.

Thank You

Downloading TA-dmarc add-on for Splunk
SHA256 checksum (ta-dmarc-add-on-for-splunk_411.tgz) b005620a0f6579323a408465deb53e0d7e8abaf9b6e4686e0d5c43f0e3ce0278 SHA256 checksum (ta-dmarc-add-on-for-splunk_400.tgz) 72b37fd57da0459678ccde26c8fbfb37a7a1ce2a965d82445235de997837b3df SHA256 checksum (ta-dmarc-add-on-for-splunk_324.tgz) 320cb3c2e8d241313897d98907b99760063e9aa8d385df18c9ef3a03622d3d0f SHA256 checksum (ta-dmarc-add-on-for-splunk_323.tgz) 494c292148cc55db43a9fab50d41ae43e54c4340dc5472e0c0954a6c55a4eec0 SHA256 checksum (ta-dmarc-add-on-for-splunk_321.tgz) 58acc4a1deaea84f41d7738c6fdd487e21ef7ce8f7f55e6eddc253ef2cdea6ec SHA256 checksum (ta-dmarc-add-on-for-splunk_320.tgz) 4fa1665aa3bccc8ffde1f31d2832d408db0270bf205a7739e8f75ab36a3faf5c SHA256 checksum (ta-dmarc-add-on-for-splunk_310.tgz) 8ae11dd67dd66e3bb1403ac861eea57eba1160f8bfd0e36ec4b349e397940459 SHA256 checksum (ta-dmarc-add-on-for-splunk_300.tgz) ba39f463446d6d67c90f25fd52f5f4878de8848a769004398ad18f55728473f6 SHA256 checksum (ta-dmarc-add-on-for-splunk_211.tgz) 18078dffbcfb46c54616de5ab6fd1f21e02323cca94b41e2a10bd185333c2c94 SHA256 checksum (ta-dmarc-add-on-for-splunk_201.tgz) 86adf8141c43c32793e4b33a5f888989db5bad680b5362165851bd6f49c9cac5 SHA256 checksum (ta-dmarc-add-on-for-splunk_200.tgz) 57e42a207510792275ed00a1092cc4d411021881e99f2377be9d5b50a5a0b2e3 SHA256 checksum (ta-dmarc-add-on-for-splunk_121.tgz) 6d3e0f633dc70a4e6d04f53fb870a30fecd585095173fb52ce2688013b6ba019 SHA256 checksum (ta-dmarc-add-on-for-splunk_120.tgz) 44cc329c8b0407542d8f4544daee3dae31f52e21a911635118bfdfa4f87c7507
To install your download
For instructions specific to your download, click the Details tab after closing this window.

Flag As Inappropriate

splunk

TA-dmarc add-on for Splunk

This app is NOT supported by Splunk. Please read about what that means for you here.
Overview
Details
TA-dmarc add-on for Splunk supports ingesting DMARC XML aggregate reports from
an IMAP/POP3 mailbox or local directory with mitigations against:

* ZIP bombs
* gzip bombs
* various XML attack vectors like billion laughs, quadratic blowup, external entity expansion and so on
* malformed reports
* false reports (alpha)

TA-dmarc add-on for Splunk

Add-on for ingesting DMARC XML aggregate reports into Splunk from an IMAP account or local directory, with mitigations against XML, GZ and ZIP-bombs.

Supported versions and platforms

Splunk version Linux Windows
8.0 Yes Yes
8.1 Yes Yes
8.2 Yes Yes
9.0 Yes Likely but untested

Additional requirements:

  • Splunk heavy forwarder instance: Because of Python dependencies Splunk Universal Forwarder is not supported
  • KVstore: used to keep track of which IMAP messages or local files have already been processed. KVstore is enabled by default on Splunk instances.

Install the TA-dmarc add-on for Splunk

Single instance Splunk deployments

  1. In Splunk, click on "Manage Apps"
  2. Click "Browse more apps", search for "TA-dmarc" and install the add-on

Distributed Splunk deployments

Instance type Supported Required Description
Search head Yes Yes Install this add-on on your search head(s) where CIM compliance of DMARC aggregate reports is required
Indexer Yes No This add-on should be installed on a heavy forwarder that does the index time parsing. There is no need to install this add-on on an indexer too.
Universal Forwarder No No This add-on is not supported on a Universal Forwarder because it requires Python
Heavy Forwarder Yes Yes Install this add-on on a heavy forwarder to ingest DMARC XML aggregate reports into Splunk.

The following table lists support for distributed deployment roles in a Splunk deployment:

Deployment role Supported Description
Search head deployer Yes Install this add-on on your search head deployer to enable CIM compliance of DMARC aggregate reports on a Search Head Cluster
Cluster Master No This add-on should be installed on a heavy forwarder that performs parsing at index time. There is no need to install this add-on on an indexer too.
Deployment Server Depends This add-on can be (1) deployed unconfigured to a client or (2) deployed preconfigured with a directory input. Due to the encrypted credentials it cannot be deployed preconfigured for IMAP inputs.

Configure inputs for TA-dmarc

The TA-dmarc supports the following input modes:

  • Read aggregate reports from an IMAP account. The add-on only ingests mails with "Report domain:" in the subject. It leaves the ingested mails on the IMAP server and keeps a record of which mails have already been processed.
  • Read aggregate reports from a directory. This can be useful to batch load the aggregate reports in non-internet-connected environments.

Directory input

TA-dmarc can watch a folder where you drop DMARC aggregate reports manually or otherwise.
It will read files with .xml, .zip or .xml.gz extention, ingest them into Splunk. Any invalid .xml, .zip or .xml.gz files are ignored.

TA-dmarc will leave files untouched in the directory: it uses internal checkpointing to skip files that have been previously ingested.

  1. Go to the add-on's configuration UI and configure a new modular input by clicking on the "Inputs" menu.
  2. Click "Create new input"
  3. Select "DMARC directory"
  4. Configure:
  5. Name: e.g. "production_dmarc_indir"
  6. Interval: how often to poll the directory where DMARC XML aggregate reports are read from (see below)
  7. Index: what Splunk index to send the aggregate reports to
  8. Directory: Location where DMARC aggregate reports should be read from
  9. Quiet time: Ignore files that have a modification time of less than n seconds ago. You can use this to prevent ingesting large files that are dropped on a network share but take some time to transfer
  10. Resolve IP: Whether or not to resolve the raw source_ip in the DMARC XML aggregate reports
  11. Click add

IMAP input

TA-dmarc can fetch DMARC aggregate report attachments from mails on an IMAP server. It will process attachments in .xml, .zip or xml.gz format and ingest them into Splunk.

TA-dmarc will leave the mails on the server: it uses internal checkpointing to skip mails that have been previously ingested.

  1. Go to the add-on's configuration UI and configure an account to authenticate with:
  2. Account Name: descriptive account name, e.g. google_dmarc_mailbox
  3. Username: the account to identify with
  4. Password: the password to authenticate with
  5. Next, go to the add-on's configuration UI and configure a new modular input by clicking on the "Inputs" menu.
  6. Click "Create new input"
  7. Select "DMARC mailbox"
  8. Configure:
  9. Name: e.g. dmarc-google
  10. Interval: how often to poll the mailserver for aggregate reports.
  11. Index: what Splunk index to send the aggregate reports to
  12. Global Account: select the account to authenticate with
  13. IMAP server: the imap server to poll
  14. Use SSL: whether or not to use an encrypted connection
  15. Resolve IP: Whether or not to resolve the row source_ip in the DMARC XML aggregate reports

DMARC aggregate reports

This add-on handles the following file formats in which aggregate reports are delivered:

  1. XML (as .xml file)
  2. ZIP (as .zip file)
  3. GZ (as .xml.gz file)

Mitigations are in place against:

  • ZIP bombs
  • gzip bombs
  • various XML attack vectors like billion laughs, quadratic blowup, external entity expansion and so on

Field mapping

From the XML sample below, these fields are created:

XML field Splunk field name Value
feedback/report_metadata/org_name rpt_metadata_org_name google.com
feedback/report_metadata/email rpt_metadata_email noreply-dmarc-support@google.com
feedback/report_metadata/extra_contact_info rpt_metadata_extra_contact_info https://support.google.com/a/answer/2466580
feedback/report_metadata/report_id rpt_metadata_report_id 13190401177475355109
feedback/report_metadata/date_range/begin rpt_metadata_date_range_begin 1506988800
feedback/report_metadata/date_range/end rpt_metadata_date_range_end 1507075199
feedback/policy_published/domain policy_published_domain example.com
feedback/policy_published/adkim policy_published_adkim r
feedback/policy_published/adpf policy_published_aspf r
feedback/policy_published/p policy_published_p none
feedback/policy_published/pct policy_published_pct 100
feedback/record/row/source_ip row_source_ip 192.0.2.78
feedback/record/row/count row_count 1
feedback/record/row/policy_evaluated/disposition row_policy_evaluated_disposition none
feedback/record/row/policy_evaluated/dkim row_policy_evaluated_dkim fail
feedback/record/row/policy_evaluated/spf row_policy_evaluated_spf fail
feedback/record/identifiers/header_from identifiers_header_from example.com
feedback/record/auth_results/spf/domain auth_result_spf_domain example.com
feedback/record/auth_results/spf/domain auth_result_spf_result fail

Authentication datamodel

Besides the fields contained in the report, additional fields are mapped from the CIM Authentication datamodel, based on the XML sample below:

Authentication datamodel field name Value
action failure
app dmarc
dest google.com
signature Use of mail-from domain example.com
signature_id 13190401177475355109
src resolved.name.if.available.test
src_ip 192.0.2.78
src_user example.com
eventtype dmarc_rua_spf_only
tag authentication, insecure

DMARC XML sample

<?xml version="1.0" encoding="UTF-8" ?>
<feedback>
  <report_metadata>
    <org_name>google.com</org_name>
    <email>noreply-dmarc-support@google.com</email>
    <extra_contact_info>https://support.google.com/a/answer/2466580</extra_contact_info>
    <report_id>13190401177475355109</report_id>
    <date_range>
      <begin>1506988800</begin>
      <end>1507075199</end>
    </date_range>
  </report_metadata>
  <policy_published>
    <domain>example.com</domain>
    <adkim>r</adkim>
    <aspf>r</aspf>
    <p>none</p>
    <sp>none</sp>
    <pct>100</pct>
  </policy_published>
  <record>
    <row>
      <source_ip>192.0.2.78</source_ip>
      <count>1</count>
      <policy_evaluated>
        <disposition>none</disposition>
        <dkim>fail</dkim>
        <spf>fail</spf>
      </policy_evaluated>
    </row>
    <identifiers>
      <header_from>example.com</header_from>
    </identifiers>
    <auth_results>
      <spf>
        <domain>example.com</domain>
        <result>fail</result>
      </spf>
    </auth_results>
  </record>
</feedback>

Advanced

The DMARC-imap input saves checkpointing data in KVstore.
To see its contents: |inputlookup ta_dmarc_checkpointer_lookup

If you want to reindex an entire mailbox, you can do so by deleting the TA-dmarc KVstore checkpointing data through this Splunk command:

|inputlookup ta_dmarc_checkpointer_lookup
|search state!="*input=dmarc_imap, server=imap.gmail.com*"
|outputlookup ta_dmarc_checkpointer_lookup`

If you want to reindex a single DMARC report, you can do so by deleting its corresponding record from KVstore:

| inputlookup ta_dmarc_checkpointer_lookup 
| search state!="*Report-ID: 3596274322387252907*" 
| outputlookup ta_dmarc_checkpointer_lookup

Reindexing a DMARC report from a directory input is left as an excercise for the reader.

Third party software credits

The following software components are used in this add-on:

  1. defusedxml version 0.5.0 by Christian Heimes
  2. IMAPClient version 1.0.2 by Menno Finlay-Smits
  3. Splunk Add-on Builder version 2.2.0 by Splunk and the third-party software it uses

Support

This is an open source project without warranty of any kind. No support is provided. However, a public repository and issue tracker are available at https://github.com/jorritfolmer/TA-dmarc

Release Notes

Version 4.1.1
Oct. 21, 2022
  • Fixed splunkbase check_python_sdk_version see issue #45
  • Fixed check_for_vulnerable_javascript_library_usage, issue #45
Version 4.0.0
Aug. 12, 2022
  • Recreated from scratch using Splunk Add-on Builder v4.1.1 to address various Splunk Cloud requirements
  • Fixed str decode exception (Thanks to George Luong for reporting the issue)
  • Fixed Py2/3 issues in POP3 input (Thanks to Constantin Oshmyan for reporting and fixing the issue)
  • Fixed POP3 uidl persistence issue (Thanks to Constantin Oshmyan for reporting and fixing the issue)
  • Fixed exception when encountering non-RFC822 items on DavMail (thanks to Diogo Silva for reporting the issue)
Version 3.2.4
Nov. 28, 2020

Fixed exception in directory input. (Thanks to Georgi Georgiev for providing a patch)

Version 3.2.3
Nov. 26, 2020
  • Added support for Splunk 8.1. (Thanks to Aaron Myers for reporting the issue)
  • Fixed exception when using add-on together with listenOnIPv6=yes (Thanks to gryphius for reporting this issue)
  • Fixed Gmail POP3 issue
Version 3.2.1
Feb. 16, 2020

Fixed resolving of IP addresses in src field.
(Thanks to Martin Wright for reporting this issue)

Version 3.2.0
Dec. 28, 2019

Added support for Splunk 8.x and Python 3.x

Version 3.1.0
Sept. 23, 2018
  • Fixed connection reset errors for large IMAP mailboxes (Thanks to Mike Kolk for the patch)
  • Lowercase keys and values in policy_published and records (Thanks to Christopher G Andrews for reporting this issue)
  • Added field in configure screen to set batch size for IMAP inputs
Version 3.0.0
May 14, 2018

Upgrading from 2.x to 3.x

The event output format changed to JSON. This is the new default because JSON is a better match for the structured aggregate reports than a flat key=value (KV) format. See for example issue #4 on Github. If you want your inputs to to keep using the KV format:

  • Disable your inputs before upgrading
  • Upgrade the add-on to version 3.x
  • Edit your inputs and change the output format to KV
  • Enable your inputs

Note that KV format output is likely to be deprecated in future versions, and enhancements like multiple XSD validation aren't available in KV output format.

Changelog

  • Added POP3 support
  • Added initial DKIM signature checking
  • Added validations against multiple XSDs
  • Added XSD validation results in a new field
  • Added JSON output to better preserve report structure, issue #4.
  • Lots of other small issues, see https://github.com/jorritfolmer/TA-dmarc/CHANGELOG.md
Version 2.1.1
Dec. 3, 2017
  • Added DMARC XML validation against DMARC XSD
  • Fixed reading reports with absolute paths from zip files. Thanks to Steve Myers for the bug report.
Version 2.0.1
Nov. 8, 2017
  • Removed forgotten use_ssl from schema following Splunk App Cert Admin review
Version 2.0.0
Nov. 6, 2017

Improved checkpointing:

  • Store individual records in KVstore instead of using a single serialized Python set.
  • Corrected misconfigured key in KVstore collection

Addressed concerns from Splunk Cert Admin:

  • enforced use of TLS for IMAP input, by removing TLS checkbox
  • ensure tmp directories are always cleaned up, by using try finally for both directory and imap inputs
  • ensure the add-on won't write outside of splunk designated directories by implementing KVstore checkpointing for DMARC directory inputs instead of using file ops to bad/, done/ or tmp/ dirs.

NOTES BEFORE UPGRADING FROM v1.x TO 2.0.0:
Because of the changes to KVstore logic, the add-on will re-index every report on IMAP, leading to duplicate events.

Version 1.2.1
Oct. 10, 2017
  • Corrected issues from Splunk Cert Admin feedback
Version 1.2.0
Oct. 9, 2017

Added non-multipart support for mail messages on IMAP


Subscribe Share

Are you a developer?

As a Splunkbase app developer, you will have access to all Splunk development resources and receive a 10GB license to build an app that will help solve use cases for customers all over the world. Splunkbase has 1000+ apps from Splunk, our partners and our community. Find an app for most any data source and user need, or simply create your own with help from our developer portal.

Follow Us:
Splunk, Splunk>,Turn Data Into Doing, Data-to-Everything, and D2E are trademarks or registered trademarks of Splunk Inc. in the United States and other countries. All other brand names,product names,or trademarks belong to their respective owners.