Adaptive Generalization Through Bridged Specialization

‘Small Data’ Is Also Crucial for Machine Learning
The most promising AI approach you’ve never heard of doesn’t need to go big

Also known as “fine-tuning,” transfer learning is helpful in settings where you have little data on the task of interest but abundant data on a related problem. The way it works is that you first train a model using a big data set and then retrain slightly using a smaller data set related to your specific problem.

Another way of thinking about the value of transfer learning is in terms of generalization. A recurring challenge in the use of AI is that models need to “generalize” beyond their training data—that is, to give good “answers” (outputs) to a more general set of “questions” (inputs) than what they were specifically trained on. Because transfer learning models work by transferring knowledge from one task to another, they are very helpful in improving generalization in the new task, even if only limited data were available.

While this article focuses on machine learning, this same technique works for humans and I’m assuming it was created for machines (AI) to replicate the human ability.

The key thing for this to work though is that the data has to be related in some way. The beauty with humans though is that the relatability can be created from a creative “weak link” which means it will probably only appears relatable to that specific person, based upon their own constructed “space of possibilities” within their mind (see Beau Lotto’s work).

But that’s exactly why it seems creative and innovative to others because they can’t see the connections that bridge the gap between these two things, thus making them relatable.

The whole point of this though is that it shows how we can all adapt in the future and discover work outside our normal domains of knowledge, by seeming similar patterns and principles they transfer between them.

Of course, the only major thing preventing this from happening is people’s biases disbelieving the person’s capacity for the work because they are approaching it from an unconventional angle than the status quo is approaching it.

In fact, as many articles have highlighted recently, this is why many great job candidates are never ever seen by employers because they don’t fit into the limited filter set defined by the job and thus are often filtered out. So exactly the same way people’s biases filter out the potential and possibility of someone being able to do something.

By Nollind Whachell

From playing within imaginary worlds to imagining a world of play.

Leave a Reply