Overview
​​
Implementing a Configuration Management Database (CMDB) is not something you just do, it is something you intentionally plan and methodically implement and consistently maintain.  Every individual Configuration Item (CI) is a crucial cog in a complex machine.  No matter the budget, no matter the timeline, no matter how skilled the engineers, a successful CMDB takes ongoing care and maintenance.  Without that intention you end up with bad data, and as most know, ridding yourself of that bad data can be an overwhelming feat. 
 
One regular CMDB maintenance activity we perform is identifying, remediating, and preventing duplicate CIs.  These are more than just a nuisance or a metric on the CMDB health dashboard, they are just as critical as missing CIs.  If your IT and Business processes can’t reference the correct CI, you’re missing out on valuable data and run the risk of making decisions based on outdated information.  

What is a duplicate CI and how do they happen? 

A duplicate CI is any two or more records in your CMDB that represent the same conceptual object.  CIs are populated via several different methods, including manual entry, import from a flat file (XLSX, Text, etc), integration with another system of record, and ServiceNow Discovery.  Every one of these has the potential to create duplicate CIs.  
 
Do tell, how does this happen, you say?  For starters, a server named DEVUNIX001 could be discovered and added to the CMDB.  Months later when that same server is renamed to COUNIX001, discovery considers it to be a new server and creates a new CI.  Another example is the IP address, where subnets can change and machines receive new IP addresses.  If you’re using IP address to match (i.e. your coalesce value) discovered machines to existing CIs, you could end up with duplicate CIs instead of updating existing records.  If the data sources that feed a CMDB are not properly configured and monitored, any one of them can cause issues.  
 
We’ve all heard the term ‘garbage in, garbage out’ and your CMDB is no exception to this rule.  It’s nearly impossible for any single system to be the original source of truth for all areas of the infrastructure.  We all work in a complex landscape of advanced technology where teams have their own management tools to support their devices, and each one of these systems is likely to become a data source for your CMDB. 
 
Perhaps you use SCCM for Windows computers and servers, Casper for Apple devices, SolarWinds for network gear, AirWatch for mobile devices, and ServiceNow Discovery for Linux machines.  It’s crucial to understand if and how the owners of these systems are keeping their data clean.  And manual entry?  I’m guessing we all know the risks of hand keyed information.  The point being your CMDB is undergoing a technical barrage of data from all sides, and without process and due diligence to manage that you will end up with duplicate CIs.

Why do I care if I have duplicates? 

The nature of a CMDB is that it is constantly being updated - weekly, daily, or even in real time.  If you have duplicate CIs, only one of them will be updated by your automated systems (discovery, integrations, etc) so data on the other CI quickly grows stale.  And why is that a problem?
 
Suppose you’re populating the assignment group on an incident based on information from the CI.  If your user selects the stale CI, this incident may get routed incorrectly.  Or what if a change request has the incorrect CI?  Automated risk calculations and approvals are based off of this stale data.  Perhaps you get lucky and this doesn’t cause a significant outage.  But as Dirty Harry asked, are you feeling lucky?
 
Whether supporting operations or analyzing trends, reliable data is paramount.  Quickly resolving an incident, troubleshooting a problem, approving a change, requesting new software, and so many other activities rely on your end users finding one specific CI record.
 
How do I stop the bleeding? 

Start by utilizing the CMDB Dashboard.  This breaks your CMDB health into three categories: completeness, compliance, and correctness.  The correctness score is further broken into three subcategories of duplicate, orphan, and staleness.  Duplicate CIs are then broken down by class.  
 
Then dig into where your fires are burning hottest:  the classes with the most duplicates.  These offer the greatest return for your effort, as it’s here you’ll likely find patterns that address tens, hundreds, or even thousands of duplicates.
 
De-duplication tasks are created automatically.  Once you’ve taken all the low hanging fruit, monitor these tasks.
 
You can also look for CIs whose discovery source is ‘Duplicate’.  
 
Finally, it’s worth noting that some duplicates are not worth the effort of remediation.  Certain classes such as load balancer pools and IP address objects don’t usually cause any major issues, and they can take a lot of time to chase down.  By prioritizing the addressing the key duplicate areas first, you’ll quickly provide qualitative benefits to your processes and CMDB consumers.

What about future duplicates? 
You don’t want to be in the business of manually remediating every single duplicate.  Often they are created much faster than a person or team could ever clean them up.  It’s important to be strategic and target the root cause.  Don’t start mopping up the water until you’ve fixed the leaky pipe. 

• Ensure your identifiers are appropriate for each class
• Utilize Identification Simulation when possible to vet new rules
  
  

Understand duplicate CIs are a natural occurrence in a CMDB.  Having duplicates doesn’t mean you’re doing it wrong; it just means you’re paying attention to the data and appreciating how it impacts your processes, or more often, how it impedes your processes. 
 
How you handle these duplicates is one of the key parts of your CMDB where you succeed or fail. 

Make sure this is someone’s job
 
This isn’t a nice-to-have, this is key to achieving next-level benefit across all of your CMDB-consuming processes.  If you went to the effort and expense of standing up a CMDB, then invest in keeping it clean and healthy.  
 
One of the tools you can use to help you with this is our CI Watch application by AJUVO.  Our CI Watch Usage identifies all records on the platform that are referencing specific CIs.  This is a critical piece of the puzzle.  
 
Deleting a duplicate without understanding where is was used can result in significant data loss.  If the stale CI was associated to operational tasks that data needs to be migrated.  If the CI has an asset associated with it, the asset will automatically be deleted.  By cleaning up one piece of bad data, you may inadvertently create another issue.  CI Watch not only finds these relationships, it also migrates them for you.  Why not take the guesswork out of your CMDB maintenance?

Or even better, let us help you! 

But what if you don’t have the time or resources or expertise to dig into and manage your duplicate CIs?  And let’s face it, the CMDB is not fun and exciting, people would rather be rolling out Virtual Agent.
 
Speak to our experts at AJUVO about CI Watch, a comprehensive solution by AJUVO to get your CMDB back on track and help keep it there.  We use our own custom applications, dedicated CMDB experts, and consultative approach to help (re)design your CMDB to support your desired business outcomes, align your existing data to achieve that value, and then provide a managed service to keep your CMDB humming along smoothly at a fraction of your normal full-time resource costs.

Overview – or, why do I care about unexpected CIs anyways?
 
Many CMDB administrators focus their time on ensuring that discovery is functioning as it should, looking for issues reported through the CMDB health dashboard and working to ensure processes are running smoothly to detect duplicates, stale CIs, etc.  However, one of the areas in the CMDB that gets very little attention is the concept of unexpected CIs.  While the existence of unexpected CIs isn’t reported on the CMDB health dashboard (today at least), it is also a very important indicator of CMDB health and can highlight areas where things may not be running smoothly.
 
What is an unexpected CI?  Before we go there, let’s do a quick review of how inheritance works in the CMDB.
 
CMDB Inheritance and the "container" classes
​ 
Remember that the CMDB classes work in an inheritance model, like a tree with cmdb_ci at the top.  As you start moving down the branches of the tree, the different classes in the middle (we will call these “container” classes) are useful as aggregation points for CI classes.  A good example of this is cmdb_ci_server, which is the branch point for cmdb_ci_linux_server, cmdb_ci_win_server, etc.  As a result, the inheritance structure (tree) looks something like this:

​Fig 1.  Sample inheritance tree for cmdb_ci_linux_server
 
 
In the CMDB inheritance model, attributes defined at one level are inherited by all classes below that definition point.  So for example, defining an attribute at cmdb_ci_computer makes it visible to cmdb_ci_server and cmdb_ci_linux_server.  Using the tree above as a guideline, the CMDB administrator can carefully plan out CI attributes so that they are inherited correctly by subclasses but do not appear in classes where they do not make sense (such as having “Service Pack” as an attribute for a network adapter).  The “container” classes such as cmdb_ci_server are very useful for providing aggregation points for defining attributes that apply to multiple child classes.
 
If we’re doing things correctly, we find that as we move down the tree and start getting more specific, classes start to have more attributes.  Those attributes are only visible at the definition point and below; so for example, kernel_release (defined in cmdb_ci_linux_server) is not visible at cmdb_ci_hardware.
 
Where we get into issues – Unexpected CIs
 
As we’ve seen above, the “container” classes are very useful in building the inheritance model, for defining parent-child relationships, and are key to instrumenting attribute inheritance.  However, while these are useful classes for CMDB schema definition, they generally are not a good place to store actual CIs, as they do not directly represent an object in the technology environment (you didn’t deploy a “hardware”, you deployed a “Linux server”).  Where we get into trouble is when we start finding CIs show up in the container classes like cmdb_ci_hardware.  So when we start seeing CIs showing up as being based in a “container” class, we call those unexpected CIs.
 
Why we care about unexpected CIs 

They may seem benign, but unexpected CIs can really mess up a lot of what should be working with the CMDB.  Some reasons why unexpected CIs are something to be concerned with include:

• We generally want CIs to appear in the most specific class that represents that CI.  This way we are sure that attributes created specific to that class are populated.  The data in the CMDB is all designed to inform processes and provide context.  These attributes are there for a reason, and not having this data harms the effectiveness of the platform overall.
  • For example, a Linux server stored in  would not have attributes for “Kernel Release” or “Operating System”, so how will you know what team to route an Incident to for this CI?  How would you even know it’s a Linux server? 
• Often times we use reference qualifiers to limit CI selection in process forms.  The reference qualifiers may unintentionally exclude CIs if they are contained in an unexpected “container” class that would not pass the qualifier.
  • For example, a Network Change ticket may only allow items that are children of , making CIs in  invisible.
• Unexpected CIs are an indicator of issues with CMDB data integrity.  Some method of populating or updating the CMDB has gone wrong, and CIs are being added or moved to the wrong class.  This could happen for any number of reasons, such as: 
  • Users are adding CIs in the wrong place manually 
  • Someone did a data import to the wrong table 
  • Discovery is not correctly performing classification of discovered CIs 
  • An integration has gone haywire 

 
Regardless of how they got there, unexpected CIs are an important indicator of CMDB health and should be on the CMDB administrator’s list of items to watch out for and remediate.
 
Commit to doing the work

Make sure you set aside the time and resources to stay on top of the unexpected, and that includes planning to catch and remediate unexpected CIs.  Unexpected CIs are not just one-off anomalies, but are an important indicator of CMDB process effectiveness and CMDB health. 
 
And remember that repatriating unexpected CIs to their rightful home isn’t necessarily as simple as just reclassifying the CI; you want to make sure that the resulting attributes for the CI are populated correctly and that you don’t create new problems due to missing attributes in the new class, etc.
 
Or even better, let us help you! 

But what if you don’t have the time or resources or expertise to dig into and manage your unexpected CIs?  And let’s face it, the CMDB is not fun and exciting, people would rather be rolling out Virtual Agent.
 
Speak to our experts at AJUVO about CI Watch, a comprehensive solution by AJUVO to get your CMDB back on track and help keep it there.  We use our own custom applications, dedicated CMDB experts, and consultative approach to help (re)design your CMDB to support your desired business outcomes, align your existing data to achieve that value, and then provide a managed service to keep your CMDB humming along smoothly at a fraction of your normal full-time resource costs.