Since my last blog, a lot of positive changes happened at Arcserve. We have acquired two companies and a lot of exciting stuff is happening in the background. Today’s blog is about UDP Archiving which is one of the acquisitions Arcserve made.
So first of all, why did Arcserve acquire an email archive solution? Let me extract some text from the press release which explains why..
“Regardless of the size or complexity of an organization, email archiving is a fundamental part of any security, IT and document management strategy. Aside from the more obvious use cases associated with long-term email retention, an advanced archiving solution must also simplify email search and recovery, and enable users to optimize email server storage,” said Andrew Smith, Senior Research Analyst, Storage Software at IDC. “Arcserve is entering a highly competitive email archiving market, but its proven backup capabilities and established reputation in the industry make archiving a valuable addition to its portfolio. These new tools should resonate well with Arcserve’s target market, particularly midsize and decentralized enterprise organizations seeking to improve compliance and reduce risk.”
Arcserve provides data protection and with UDP archiving we are adding data management into the mix. UDP itself had already a file archive solution in UDP, but email archive was the missing component until now.
The main reasons for using email archiving could be different per company, some companies use archiving for storage efficiency, other companies use it for compliance and regulation reasons or even both. UDP Archiving provides companies with the following functionality :
- Minimise risk from email outage, when the primary email system (whether that is cloud based or on premise) is unavailable, users can still access their archived emails using the archiving solutions. As emails are collected during the sent and receive process, the end users have access to all the emails upon the production mail service went unavailable.
- Meet legal and compliancy requirements, UDP Archiving provides reporting, retention management, role based access and audit trails functionality out of the box
- Legal Hold management preserve email records as unalterable records
- Email lifecycle management, UDP archiving collects sent and received emails from the popular email systems (Office 365, Exchange, Gmail, Lotus Notes etc) and removes them automatically at the end of their retention, which is set in the policies.
- Storage efficiency, email administrators can set policies on their production email systems to decrease the amount of email stored on the servers. Meanwhile the UDP Archiving system uses a single instance storage to store the emails. This means that duplicate emails and / or attachments are only stored once in the archive.
- Muli-tenancy, UDP Archiving is created with multi tenancy in mind, administrators or service providers can setup the archiver for multiple domains and customers.
The auditors and employees are provided with simple and advanced search capabilities, ability to save search queries, and adding tags and notes. Employees have also access to the outlook plugin which I will explain later in the post.
How does the archiving solution work?
UDP Archiving is a very easy to implement solution, the solution is implement as a virtual appliance in VMware / Hyper-V or Amazon Machine Image (AMI)
Once installed and configured the UDP Archiving preserves every email as a record immediately after it is sent or received.
Depending on the rules set these emails are then stored into the archive. Emails will be purged if these meet the criteria set in the exception rules or when the retention time is met.
UDP Archiving support many email solutions like Microsoft Exchange, Office 365, Lotus notes, Google G suite etc. Both Exchange and Office 365 will use a journaling configuration, and other mail solutions are configured using either a forwarding or archive function.
The base image requirements are:
- 2 logical CPU (2.5Ghz or higher is recommended)
- 8 GB ram
- Minimum 100 GB storage
These are the minimum starting requirements, obviously when sizing your archive for your email environment these will change.
Now, let’s get into the product itself using a scenario from my local lab
Currently, in my lab I have the UDP Archiving appliance installed on a VMWare workstation, the current email system is Exchange 2013. The end users are using outlook 2013. Importing the appliance in VMware will provide a wizard to configure the appliance. Using VMware workstation this is not available, however making sure that the appliance is configured using a NIC with DHCP will grants access to the appliance setup wizard. In my case I used putty SSH into the appliance after it booted and change the DHCP address to a static address.
The UDP Archiving solution has 4 types of accounts;
- Superadmin – manage the appliance and domains
- Administrator – manage access and policies
- Auditor – access to all messages for legal, regulatory, corporate governance and HR reviews
- User / employee – access to own sent and received messages through web or outlook plugin
We will go through these 4 account types step by step using a new setup and configure the appliance for use with my Exchange server. To configure Exchange to use with UDP Archiving, I will create follow up post.
After the initial setup of the appliance, you will be presented with a small wizard setup;
First of all set the language to English
Accept the license agreement
And configure the connection details, for my lab environment I created a DNS entry in my local DNS server called udparchiving.arcserve.lab and use a static IP address.
When using DHCP, use the DHCP IP address in this section, you can change to a static IP address afterwards.
Set the time zone and date format and you can login into the appliance
The default login for this account is superadmin@archiver with the default password.
The superadmin account is used to manage and monitor the appliance itself and to setup multiple email domains. When you log in for the first time you will see the health screen
From here you view the performance of the archiving appliance.
Server Configuration: Monitor the CPU and memory usage, if one of these is constantly running high than you can add more resources to the appliance.
Processed Email Counts: is the number of processed emails for all domains combined.
Projected Storage Requirements: Shows you the storage trends and gives the Superadmin insight into how the archive is growing. These are all calculated based on historical metrics calculated in the archive
Message Disposition: Received messages are all messages coming into the archive. Duplicated messages are based on Single Instance Storage (SIS). If an employee sends a bulk email to 10 other employees, it will be stored one time and the rest shown as “Duplicated Messages”
Storage: Archive Size: Size of messages in storage after compression and SIS. Disk Usage shows used and available. Archive Status ties directly to the configuration page. Periodic Purge is based on the retention policy of the messages. The system will remove messages at their end of life.
Once logged in, the first thing I recommend to do is to change the default passwords, first lets change the superadmin password:
Click the settings menu in the superadmin menu in the right top corner
Change the password and submit.
Some management tasks can be done using a SSH into the appliance. The account for this is udp_admin. I would highly recommend to change the password for this account too:
Go to the configuration page and in the right bottom access control widget where you can change the password for this account
Now the passwords are set, you can start configuring the appliance and the domains.
Lets have a look at the configuration screen:
The settings widget displays the FQDN and IP parameters you have set during the initial wizard, you can change these if needed.
Most important is the SMTP forwarding address, this is the address used to forward email from a mail server to the archiving appliance. The proper format is always: archive@FQDN or in my scenario firstname.lastname@example.org
If needed customers and MSP can add SSL certificates in the next widget.
The email retention is a global retention for the appliance which will be overruled by the retention set by the email domain administrators. If they do not set a retention policy the default set here will be used instead.
Additional storage space can be added in the increase storage data widget, simply add a new virtual disk to the appliance and click check disk, if a volume is available you can check the check LVM and increase the storage space.
The next widget is for system maintenance, where superadmins can start / stop / restart services and shutdown the appliance.
And lastly, the access control where you can change the password for udp_admin as mentioned earlier in this post.
Domain and profiles:
Now this is set the superadmin can add the domain and the profiles with it.
Before connecting to a mails server, the domain should be added to the archiving appliance, this tells the appliance to accept the messages from these domains and reject any messages from domains not listed here.
To add the domain simply go to the domain tab and click add domain
Add you domain name and click add
Next step is to create a profile, a profile describes the entity being archived and allows you to map multiple Domains to entity or division.
To add a profile, go to the profile section and click add a profile
Enter the profile name and contact details of the main administrator of this organisation
Note, a single profile can have multiple domains mapped. If multiple domains are mapped to a profile, the admin accounts mapped to this profile can manage all of the mapped domains.
And the last step is to add a master admin account for the domain(s). Administrator are added to Profiles. More than one Administrator can be added to each Domain.
The superadmin can create multiple Admins per profile or create one Admin who can add more admins when they login to access their profile.
Click add an administrator
Add the details of the domain administrator and select the profile to map it to this admin account.
An admin manages a profile with one or more domains. They set policies and access rights but have no access to any messages for privacy and security reasons.
The dashboard gives them daily and monthly statistics on the archive. They can review trends and performance.
Archive accounting provides insight into each email address in the archive and domains. Admins can see how many messages an employee has and other details on size and the date range in the archive.
Note: both the admin and superadmin have no access to the archived messages, only the auditor can see all the messages and the employee can only see his own messages
The admin has the ability to view full audit log, all the activity by users is recorded, furthermore a search can be done and the audit log can be exported as a csv file.
The admin grants access to the archive for the domain(s) he is responsible for. He/she can manually add other users or configure LDAP authentication for employees. Normally I only create the audit user and additional admin users manually and all employee’s will use LDAP to authenticate.
Auditor and admin roles are always created manually.
In my example, I only created an audit user and let the employees authenticate against my active directory LDAP.
Groups allows one or more employees to access one or email accounts that exist in the archive, for example when an employee leaves to grant his manager access to that email archive.
Another example for groups, is when using a group email address like sales or info and grant the department access to this archive.
Employees can be granted access to archive using LDAP. It is important to know that every user in the specified Organizational Unit (OU) is granted only employee access. Best practise is to have your admin and audit accounts in different OU’s,
You can add multiple OUs entries in this section.
To configure LDAP use the add an entry
Fill in the details for
- LDAP Type: currently only Active Directory is supported, more to follow later
- Description: Any description the customer wants
- Domain: Select the domain previously added to UDP Archiving which associates with this LDAP configuration.
- LDAP Host: FQDN or IP of a domain controller
- LDAP Base DN: OU is the organization unit which contains all of the users. If a company only wants to archive some of the employees they can setup a new OU with just the employees being archived (can be set in Journaling rules too). OU=(domain of this group or whole company as in example above), DC= company domain,, DC=.com (extension)
- LDAP Bind DN: The administrator bind DN is the user name and password configured for LDAP authentication. The administrator bind DN is used only for querying the directory server and so this user must have privileges to search the directory.
- LDAP Bind Password: This is the password for the account Bind DN set in the previous step
You can test the authentication by testing an account which resides inside the specified OU. When all done, save the query
By default LDAP is disabled, when the OU is specified and you want to grant archives access to the employees, you will need to enable LDAP by clicking on the enable LDAP button:
Next step is to set the rules for archiving, and this is something that will need internal discussion with the departments and have to make sure it complies to regulations set by the government or company or others.
Important to know is that rules set, will start once applied but are not taking effect from a date from the past.
The first set of rules you can set are the exception rules, these allows you to prevent messages that meeting criteria (or set of criteria’s) from being archived.
Click add an exception to create a rule:
Fill in the details and click add. Once you add a rule you do need to click the apply changes to get the rule applied to the system.
The most important part of archiving is done in the retention rule setting, this is where you as a company define the retention of the archived emails, if this is not set, the default retention set by the superadmin will be used instead.
Personally, I would set first a rule for the entire domain with a life span of the archived emails, for instance 10 years .
Important to know is that the default purging of archived emails is always the longest retention time. So, for example I set my domain rule for 10 years and if I create a rule for the CFO to keep his emails 15 years, then the emails for this address will be kept 15 years in the archive.
Again, once the rules are added make sure you apply the changes to make these active.
Lastly, legal hold, this prevent messages from being purged for specific employees on hold. If an employee’s email address is in a message, it will be held until the hold is release. The message would then be eligible for purging at the end of their retention period.
Auditors have special permissions and are allowed to see all archived emails, this person often is appointed by the organisation itself and could be HR or a GDPR officer. Auditors can search through the emails, download emails and make notes or tag the emails. The notes and tags are only visible for that auditor account. Important to know, the emails itself are not modified whatsoever!
The first screen for the auditor is the search screen.
The basic search will provide quick and basic search capabilities for the auditor
Add text to the search bar and click search. Wildcard, Boolean searches, comma separated text, email addresses and more can be used. They can use the option to “Save Search” in blue on search bar line.
The results will be shown and you can click teh subject line once to review messages inline and twice for full screen
- Tags: On bottom right you can select messages and create or apply an existing tag. As you type existing tags appear. You can also see the reference tab for existing tags. Tags are used to group messages together (virtual folder). In Advanced Search you can search on tags. Messages can have more than one tag. An icon will display in the right columns. Hovering over will show the tags applied.
- Notes: This option is the same as tags and a field displays when you open a message. Also displayed in references and can be searched in Advanced Search. An icon will display in the right columns. Hovering over will show the notes applied.
- Download: Auditor can download a single message, selected messages or all messages as .eml files, Download to zip files. Downloaded messages can be opened in Outlook or other mail clients.
- PDF: Select PDF to download messages as PDF files
- Print: When a message is opened, you can send it directly to a printer
Advanced search allows an auditor to perform more complex searches such as:
- From email and To email
- Multiple email addresses can be added and separated by a comma.
- Ex: One FROM address and two TO addresses = all emails from one person to the other two TO addresses
- Text field to search on subject line. Boolean, wildcards and more can be used
- Subject and Body
- Same as subject but adds body text and text in attachments
- Search on all tags and find all grouped messages with the same tag
- Same as tags
- Select attachment type to return only messages with that attachment
- Date FROM and date TO
- Select the range to search
- Save Advanced Search
- Return after a search to save all of the criteria
By clicking the subject line in a messages once will display the message inline. The Auditor can apply previously created notes or create a new one for future use. Other options include viewing detailed header information and direct printing.
The auditor can also look at saved searches, this will displays a list of the searches that have been saved, clicking one will launch the search and display the results;
The references tab show all employees on legal hold, available tags and notes
Lastly, the auditor has access to the audit logs same as the administrator previously mentioned in this post.
Employees have the same view as an Auditor with full access to their own messages. They cannot see tags and notes created by an Auditor and can create their own to group message types.
Groups is also only available to an employee. If they have access to group messages they can select the group and view all messages.
Additionally, employees have also access to an outlook plugin, this is an easy to install plugin and appears under Add-Ins in outlook;
There are 2 options in this plugin in:
First of all Search, this will launch a search screen with two tabs;
The settings tab is configured once with the FQDN of the UDP Archiving system, email address and password. If LDAP is enabled, the LDAP username and password is used.
The second tab is for searching the archive. The Employee can provide their criteria and perform a search.
The results will display and the employee can select which messages to import to Outlook and click the imported selected button to import them.
When they import the messages, they are placed in a folder named: UDP Archiving. From here the employee can manage them as they would any other email in Outlook.
The second option is the Arcserve Icon, clicking this will take the employee directly to the webui login as another option to access the archive.
As you can see the UDP archiving solutions is a comprehensive and very easy to solution for companies looking for email archiving whether it is for compliancy, storage efficiency or both.
I will create some follow up posts how to configure for example Exchange and Office 365 to work together with the Arcserve UDP Archiving.
Lastly I would like to thank Steve Catanzano for helping me and providing me input for this blog, you’re a Rockstar mate!