This section explains how and why we select new models + model types for forecasting risk uncertainty on your schedule.
Context
Every schedule is different, we have multiple models available that we can apply to to each client’s schedules. We have introduced an approach to quantitatively select the best model for any particular schedule based on current performance. When assessing the current performance on a project we can determine if planning and performance is better or worse than that of the client’s training data set (i.e. past projects) and then select a pre-trained model that is more representative of current project performance.
Why change model?
We have trained multiple models with slightly different configurations or tweaks to the data they are trained with. This means that some models will perform better for specific industries, schedule formats, or project types. We do this because it’s in our client’s interest and will continue testing to see which models perform the best.
How - The process we've used matches the schedule to the model that provides the best accuracy. We can then employ the model that performs best on the client’s current schedule. Below we see the first model has the lowest average error. This does not mean that the other models are bad, just that they are less representative of the clients current project.
Model Types
We primarily use three model types at nPlan as follows:
- MLPs (Multi-layered Perceptron - focusing at the activity level, some correlations) - reads data from rows and columns.
- GNNs (Graph Neural Network - looking at activities, edge cases, links, correlations etc) - reads data from graphs constructed of nodes and edges. A social media graph might look like the image below.
- Ensemble modeling is a process where multiple diverse models are created to predict an outcome, either by using many different modeling algorithms or using different training data sets.
The model we choose will be the one most suitable for the project with the least amount of error.