In our first article, we examined how relying on file based physical network documentation creates inefficiencies for the business. In our second article, we explored different strategies for converting file based documentation into structured data, ranging from purely manual and automated methods to balanced approaches that smartly leverage both automation and human expertise.

So, you initiated the data transformation journey. You endured the project. It was a demanding process filled with obstacles, but you pushed through because you knew that if done successfully, you would harvest all the benefits of having a structured network inventory.

Looking back, you realize that the journey taught you several important lessons:

• Organizing the input. Your file-based documentation consisted of tens of thousands of files, including semi-structured CAD drawings, PDF files, Excel spreadsheets, and even handwritten notes. Now you understand how important it was to establish a comprehensive overview of every existing file, its format, and its version. You wanted to ignore redundant and obsolete engineering drawings, but you did want to reduce the risk of ignoring documentation that remained relevant.
• The multitude of documentation standards. Once the relevant files were identified, it became clear they were drafted according to several different types of standards, often dependent on the vendor that produced the as-built documentation. For each documentation standard, you created a set of rules for how the content of those files should be processed. This setting up of rules was a time and energy consuming task, but it was critically important since out of the box algorithms were customized based on those rules. Now you understand it paid off to put effort into this task because the more effort was invested in defining robust rules, the higher the automation rate of data processing that was achieved.
• Data processing. The strategic decision to approach the project with a tool that combined a high level of automation with human in the loop validation proved to be correct. You loaded an original file and initiated the first step of processing. The content of the engineering drawings was automatically processed: for example, circles were converted to points, duplicate lines were removed, and lines that were too short or too long were snapped to points representing network nodes. It was impressive to witness how drawings, which were previously only human readable, were being converted into machine readable data, forming a geometrically and topologically correct network. Furthermore, all discrepancies were identified and highlighted automatically. From here on, human intervention worked hand in hand with the automation algorithm. For each specific problematic case identified, a human instructed the algorithm how to handle it. In rare situations where even that did not help because, for example, a field visit was required, it was decided to mark those cases with a short note and migrate that along with the data into the network information system for further inspection in the future.
• The moving target. Because of the sheer number of files that needed to be processed, the data transformation journey took months to complete. In the meantime, the network did not stop changing. While you were processing the data, new as-built documentation was generated and submitted to be processed in your project. You had to find ways to integrate those changes before the final migration into your network information system took place. You also remember you were forced to make a difficult strategic choice: to migrate the processed data into your network information system area by area or attempt a big bang migration for the entire network at once. There are pros and cons for each approach, but looking back, you are glad you navigated this critical decision correctly.
• The team factor. Before the project had started, you wondered how much time your team would need to dedicate to the project. You wanted to minimize this so they could focus on their daily work. Now you know their know-how and the unwritten knowledge they had of the network made a big difference to the success of the project. Their load, however, was significantly reduced by bringing in an experienced external team that had successfully completed such data transformation projects in the past. Their experience brought a lot of value to the table and helped you overcome several make-or-break challenges you faced during the project.

The win

Now you also realize that a data transformation project is, at its core, a data cleansing project. Even if you had hired a dedicated external company in the past to verify the quality of your as-built documentation, many data discrepancies were still identified and successfully resolved during this data transformation journey. This project did more than just transform the data: it was like putting your documentation in a washing machine to clean it. This experience was humbling to a certain extent, as it challenged the previous assumption that the documentation was of higher quality.

Now that those challenges are behind you, there is no feeling quite like the day the new system goes live. After months of hard work, your modern network information system is up and running. The legacy data is cleaned and ingested. Technicians are logging in. The entire outside plant network is visualized on a single map.

It feels like a big victory. For the first time, you feel you are in control.

• You can tell the board exactly what the financial value of your fiber network is.
• Your AI strategy can actually be implemented because you finally have structured data.
• The sales team finally knows exactly where they can market services, turning previously invisible assets into revenue.

                                        Figure 1: Has your digital transformation been fully completed?

--> But what about operations, such as planning, troubleshooting, provisioning, and maintenance?

--> Have they really benefited as you assumed they would?

--> Has your digital transformation been fully completed?

In our next post, we will explore the gap between the data transformation promise and the daily reality of an OSP engineer.