MIT scientists have recently identified a cause of concern for neural networks, and that is “overinterpretation,” where algorithms make confident predictions based on details that don’t make sense to humans, like random patterns or image borders. This could be a potential problem for self-driving cars and medical diagnostics which are in high-stakes environments.
"While it may seem that the model is the likely culprit here, the datasets are more likely to blame."
This may mean creating datasets in more controlled environments. Currently, it’s just pictures that are extracted from public domains that are then classified. But if you want to do object identification, for example, it might be necessary to train models with objects with an uninformative background.
For more content around the Data World delivered to your mailbox, consider subscribing.