xingtwittersharerefreshplay-buttonpicture as pdfLogo_DPM_SCHRIFTZUGinstagram icon blackShapeGroup 3 Copy 2Group 2 Copydepartment_productdepartment_datascienceuserclosebasic clockblogShapearrows slim right copy 3arrows slim right copy 3arrows slim right copy 3
autoRetouch - The way to full automation of Ghost Mannequin Effect (with GANs)

autoretouch GmbH

autoRetouch - The way to full automation of Ghost Mannequin Effect (with GANs)

With autoRetouch we were able to overcome the limits of the status quo: The goal was to completely automate the production of the so-called "Ghosted Mannequin Pictures". With GANs this should be possible for the first time. If you want to learn how to work with cutting-edge technologies in the field of data science, then you've come to the right place!

In the field of eCommerce, product images are the only point of contact between a customer and the product. For this reason, the images must create a perfect impression on the customer. This is especially true when it comes to the fashion industry, where the appearance of a product is one of the strongest driving factors for purchase decisions.

For this reason, image processing can be considered a key element of eCommerce for the fashion sector. High quality standards while maintaining high efficiency and reducing costs at the same time is therefore a major challenge. For this purpose, autoRetouch has developed a platform that contributes to automated image processing, where users can pre-define their own workflows to benefit from the AI capabilities of the software. This is the first time that you have the possibility to edit your images with minimal effort and without repetitive workflows.

[...] while the artistry in the creation of the product pictures themselves has evolved, the retouching process is pretty much the same as it was years ago.

From autoRetouch blog

One of the problems of fashion e-commerce is that a simple photo of the clothing item is not enough. Customers also want to get a taste of how their favorite piece of clothing will fit. Unless the clothing is to be worn by models, which can distract attention from the item itself, at least the contours of a body should still be recognizable. In the so-called “Ghost Mannequin Product Pictures”, only the clothing is visible and it seems to be worn by a ghost.

This kind of representation seems to be very effective in triggering purchase decisions. Unfortunately, the manual process of creating the images is complex and time consuming. autoRetouch has developed an optimized workflow especially for this purpose, reducing the effort of post-processing to the joining of two images. Here in their blog the process is explained in more detail.


autoRetouch Workflow for Ghost Mannequin Effect: AI-controlled segmentation and manual merging of a front and an inner image
autoRetouch Workflow for Ghost Mannequin Effect: AI-controlled segmentation and manual merging of a front and an inner image

What was the way there?

We tried to completely automate the process of developing Ghost Mannequin Product Pictures, i.e. to develop an AI that can combine two images in such a way that the resulting picture looks realistic afterwards.

We got to the root of the matter and faced a number of exciting challenges:

  • We worked with a large amount of ultra-high-resolution images, which were characterized by very fine details, such as patterns or brand logos;
  • There was no established technology to solve this kind of problem;
  • The data available was incomplete: we knew the input, the target, but not the transformations in between.

  • We had no guarantee for the success of the platform and therefore it was necessary to manage the project appropriately, both within our team and with the customer.

In summary: A project of great demand, full of uncertainties, which requires a lot of experimentation and a mature management and communication strategy.


Tensorboard offers many features to supervise experiments in real time
Tensorboard offers many features to supervise experiments in real time

How did we approach it?

In data science, it can always happen that projects do not work out the way you want them to. This is not necessarily because you are on the wrong track, as implementation is often difficult and the obvious way is not necessarily the right one. This is especially true when it comes to unexplored areas in advanced technology.

This is what we have focused on from the very beginning:

  • To create a setting in which various experiments could be carried out quickly (flexible data pipeline, monitoring of educational processes, scaling of computing power).
  • Frequent and honest communication with the autoRetouch team to keep them up to date and to keep their suggestions and expectations in mind was essential for the successful implementation.

Transparency has a very special value in a project with a lot of variables. During development it was therefore essential to discuss with the autoRetouch project team and to communicate possible failures. We were lucky to have colleagues with an affinity for data who knew exactly how the system works.


A rough scheme of the chosen architecture pix2pix
A rough scheme of the chosen architecture pix2pix

How does the finished technical solution look like?

In our opinion, Generative Adversarial Networks (short GANs) provided the right framework to address the problem. We identified the pix2pix architecture as a good method for our challenge, because we wanted to develop a network that learns how to fill the missing parts in the front image with information from the image from the inside out. In practice, we were coaching two networks simultaneously:

  • A generator that creates the filled images, with the goal of achieving the most realistic result as possible;
  • A discriminator, which observes the results of the generator and tries to separate fakes from real images.
  • We implemented the entire model - including the pre-processing pipeline - in Tensorflow and trained it on the Google AI platform.

The "best in class" that could generate our model (without overfitting!)
The "best in class" that could generate our model (without overfitting!)
And some where the fine details are not yet visible.
And some where the fine details are not yet visible.

What are the results?

Although the project was not a one-way process, but rather an experimental campaign with an uncertain outcome, we achieved a lot within a month.

Now we can:

  • create realistic ghosted pictures. This concerns for example the outlines, patterns and shadows of the pictures, but some fine details like the logos and labels of the brands are still missing.
  • Identify major sources of error (inconsistency between input and target images) and set a roadmap for further experiments to solve such problems.

In this way, our customer was also able to get an idea of the actual effort required to solve the problem within the required quality standards and to adjust his own product roadmap accordingly.


AndreasFranz

Projects with great uncertainty and many experiments require a high level of communication and mutual understanding. We at DieProduktMacher always try to see things from the perspective of our customers. I am convinced that this approach is the right one - especially for data projects.

Andreas Franz Data Scientist


Kunde
autoretouch GmbH
Link

https://autoretouch.com/

Duration

1 Monat

Participants

3

Location

Stuttgart

Weitere Projekte & Produkte

Weitere Projekte & Produkte



facebook

Wird geladen…

twitter

Why #diversity matters in #product design? Buildin…

instagram

Wird geladen…

New Member of the Team: Henry Simanek, Product Manager
blog

New Member of the Team: Henry Simanek, Product Man…