{"id":1175,"date":"2019-08-03T17:01:20","date_gmt":"2019-08-03T20:01:20","guid":{"rendered":"http:\/\/web.inf.ufpr.br\/vri\/?page_id=1175"},"modified":"2022-01-11T02:21:30","modified_gmt":"2022-01-11T05:21:30","slug":"laroca2018robust","status":"publish","type":"page","link":"https:\/\/web.inf.ufpr.br\/vri\/publications\/laroca2018robust\/","title":{"rendered":"A Robust Real-Time Automatic License Plate Recognition Based on the YOLO Detector"},"content":{"rendered":"\n<div class=\"wp-block-image\">\n<figure class=\"aligncenter\"><img fetchpriority=\"high\" decoding=\"async\" class=\"alignnone wp-image-1176\" src=\"http:\/\/web.inf.ufpr.br\/vri\/wp-content\/uploads\/sites\/7\/2019\/08\/pipeline-1024x336.jpg\" alt=\"The proposed ALPR pipeline\" width=\"800\" height=\"263\" srcset=\"https:\/\/web.inf.ufpr.br\/vri\/wp-content\/uploads\/sites\/7\/2019\/08\/pipeline-1024x336.jpg 1024w, https:\/\/web.inf.ufpr.br\/vri\/wp-content\/uploads\/sites\/7\/2019\/08\/pipeline-300x98.jpg 300w, https:\/\/web.inf.ufpr.br\/vri\/wp-content\/uploads\/sites\/7\/2019\/08\/pipeline-768x252.jpg 768w, https:\/\/web.inf.ufpr.br\/vri\/wp-content\/uploads\/sites\/7\/2019\/08\/pipeline-360x118.jpg 360w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/><\/figure>\n<\/div>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"paper-information\"><b>1. Paper Information<\/b><\/h3>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"abstract\"><b>1.1. Authors<\/b><\/h4>\n\n\n\n<p>Rayson Laroca, Evair Severo, Luiz A. Zanlorensi, Luiz S. Oliveira, Gabriel R. Gon\u00e7alves, William Robson Schwartz, David Menotti.<\/p>\n\n\n\n<h4><b>1.2. Abstract<\/b><\/h4>\n\n\n\n<p><i>Automatic License Plate Recognition (ALPR) has been a frequent topic of research due to many practical applications. However, many of the current solutions are still not robust in real-world situations, commonly depending on many constraints. This paper presents a robust and efficient ALPR system based on the state-of-the-art YOLO object detector. The Convolutional Neural Networks (CNNs) are trained and fine-tuned for each ALPR stage so that they are robust under different conditions (e.g., variations in camera, lighting and background). Especially for character segmentation and recognition, we design a two-stage approach employing simple data augmentation tricks such as inverted License Plates (LPs) and flipped characters. The resulting ALPR approach achieved impressive results in two datasets. First, in the SSIG dataset, composed of 2,000 frames from 101 vehicle videos, our system achieved a recognition rate of 93.53% and 47 frames per second (FPS), performing better than both Sighthound and OpenALPR commercial systems (89.80% and 93.03%, respectively) and considerably outperforming previous results (81.80%). Second, targeting a more realistic scenario, we introduce a larger public dataset, called UFPR- ALPR dataset, designed to ALPR. This dataset contains 150 videos and 4,500 frames captured when both camera and vehicles are moving and also contains different types of vehicles (i.e., cars, motorcycles, buses and trucks). In the proposed dataset, the trial versions of commercial systems achieved recognition rates below 70%. On the other hand, our system performed better, with a recognition rate of 78.33% and 35 FPS.<\/i><\/p>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"references\"><b>1.3. Citation<\/b><\/h4>\n\n\n\n<p>\nIf you use our trained models or the UFPR-ALPR dataset in your research, please cite our paper:<\/p>\n\n\n<ul>\n<li>R. Laroca,&nbsp;<span class=\"tadv-format-panel nobr\">E. Severo,<\/span>&nbsp;<span class=\"tadv-format-panel nobr\">L. A. Zanlorensi,<\/span>&nbsp;<span class=\"tadv-format-panel nobr\">L. S. Oliveira,<\/span>&nbsp;<span class=\"tadv-format-panel nobr\">G. R. Gon\u00e7alves,<\/span>&nbsp;<span class=\"tadv-format-panel nobr\">W. R. Schwartz,<\/span>&nbsp;<span class=\"tadv-format-panel nobr\">D. Menotti,<\/span>&nbsp;<em>\u201cA Robust Real-Time Automatic License Plate Recognition Based on the YOLO Detector,\u201d<\/em>&nbsp;in International Joint Conference on Neural&nbsp;<span class=\"tadv-format-panel nobr\">Networks (IJCNN),<\/span>&nbsp;<span class=\"tadv-format-panel nobr\">July 2018, pp. 1\u201310.<\/span>&nbsp;[<strong><a href=\"https:\/\/ieeexplore.ieee.org\/document\/8489629\" target=\"_blank\" rel=\"noopener noreferrer\">IEEE Xplore<\/a><\/strong>] [<a href=\"http:\/\/web.inf.ufpr.br\/vri\/wp-content\/uploads\/sites\/7\/2019\/08\/laroca2018robust.pdf\" target=\"_blank\" rel=\"noopener noreferrer\"><strong>PDF<\/strong><\/a>] [<strong><a href=\"https:\/\/raysonlaroca.github.io\/bibtex\/laroca2018robust.txt\" target=\"_blank\" rel=\"noopener noreferrer\">BibTeX<\/a><\/strong>] [<a href=\"http:\/\/web.inf.ufpr.br\/vri\/wp-content\/uploads\/sites\/7\/2019\/08\/laroca2018robust-1.pdf\" target=\"_blank\" rel=\"noopener noreferrer\"><strong>Presentation<\/strong><\/a>] <b>[<a href=\"https:\/\/news.developer.nvidia.com\/researchers-develop-ai-system-for-license-plate-recognition\/\" target=\"_blank\" rel=\"noopener noreferrer\">NVIDIA News Center<\/a>]<\/b> [<a href=\"https:\/\/www.youtube.com\/watch?v=XALyMj_hBvU\" target=\"_blank\" rel=\"noopener noreferrer\"><strong>Video Demonstration<\/strong><\/a>]<\/li>\n<\/ul>\n<p class=\"indent\">You may also be interested in the&nbsp;<span style=\"color: #ff0000\"><b>extended version<\/b><\/span>&nbsp;of this paper, where we considerably improved our system:<\/p>\n<ul>\n<li>R. Laroca, L. A. Zanlorensi, G. R. Gon\u00e7alves, E. Todt, W. R. Schwartz, D. Menotti,&nbsp;<em>\u201cAn Efficient and Layout-Independent Automatic License Plate Recognition System Based on the YOLO Detector,\u201d<\/em> IET Intelligent Transport Systems, vol. 15, no. 4, pp. 483-503, 2021. [<a href=\"http:\/\/web.inf.ufpr.br\/vri\/publications\/layout-independent-alpr\/\" target=\"_blank\" rel=\"noopener noreferrer\">Webpage<\/a>] [<a href=\"http:\/\/doi.org\/10.1049\/itr2.12030\" target=\"_blank\" rel=\"noopener\">Wiley<\/a>] [<a href=\"https:\/\/web.inf.ufpr.br\/vri\/wp-content\/uploads\/sites\/7\/2021\/05\/laroca2021efficient-published.pdf\" target=\"_blank\" rel=\"noopener noreferrer\">PDF<\/a>] [<a href=\"https:\/\/raysonlaroca.github.io\/bibtex\/laroca2021efficient.txt\" target=\"_blank\" rel=\"noreferrer noopener\" aria-label=\" (opens in a new tab)\">BibTeX<\/a>]<\/li>\n<\/ul>\n<h3 id=\"downloads\"><b>2. Downloads&nbsp;<i><\/i><\/b><\/h3>\n\n\n<h4 class=\"wp-block-heading\" id=\"proposed-system\"><b>2.1. Proposed ALPR System<\/b><\/h4>\n\n\n\n<p class=\"indent\">We presented a robust real-time end-to-end ALPR system using the state-of-the-art YOLO object detection CNNs. We trained a network for each ALPR stage, except for the character recognition where letters and digits are recognized separately (with two distinct CNNs).<\/p>\n\n\n\n<p class=\"indent\">The bottleneck of ALPR systems is the character segmentation and recognition stages. In this sense, we performed several approaches to increase recognition rates in both stages, such as data augmentation to simulate LPs from other vehicle\u2019s categories and to increase characters with few instances in the training set. Although simple, these strategies were essential to accomplish outstanding results.<\/p>\n\n\n\n<p class=\"indent\">The&nbsp;<a href=\"https:\/\/github.com\/pjreddie\/darknet\/\" target=\"_blank\" rel=\"noopener noreferrer\">Darknet framework<\/a>&nbsp;was employed to train and test our networks.<\/p>\n\n\n\n<p class=\"indent\">The architectures and weights can be&nbsp;<span class=\"nobr\">downloaded&nbsp;<a href=\"http:\/\/www.inf.ufpr.br\/vri\/databases\/SupplementaryFiles.zip\"><b>here<\/b><\/a>.<\/span><\/p>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"dataset\"><b>2.2. UFPR-ALPR Dataset<\/b><\/h4>\n\n\n\n<p class=\"indent\">We also introduced a public dataset for ALPR that includes 4,500 fully annotated images (with over 30,000 LP characters) from 150 vehicles in real-world scenarios where both the vehicle and the camera (inside another vehicle) are moving. Compared to the largest Brazilian dataset (SSIG) for this task, our dataset has more than twice the images and contains a larger variety in different aspects.<\/p>\n\n\n\n<p class=\"indent\">Full details regarding the dataset, including download instructions, can be&nbsp;<span class=\"tadv-format-panel nobr\">seen&nbsp;<a rel=\"noopener noreferrer\" href=\"https:\/\/web.inf.ufpr.br\/vri\/databases\/ufpr-alpr\/\" target=\"_blank\"><b>here<\/b><\/a>.<\/span><\/p>\n\n\n\n<h3 class=\"wp-block-heading\"><strong>3. Additional Results<\/strong><\/h3>\n\n\n<p>To enable comparisons with approaches designed specifically for cars (i.e., approaches that do not work for motorcycles), here we separately report the recognition rates obtained on images of cars and motorcycles (see table below). All authors who downloaded the dataset were notified of this update on August 2, 2019.<\/p>\n<table style=\"border-collapse: collapse;width: 95%;height: 195px;text-align: center;margin-left: auto;margin-right: auto\">\n<tbody>\n<tr style=\"height: 44px\">\n<td style=\"width: 28.7736%;text-align: center;height: 44px;vertical-align: middle\">ALPR System<\/td>\n<td style=\"width: 21.2264%;text-align: center;height: 44px;vertical-align: middle\">Cars<\/td>\n<td style=\"width: 25%;text-align: center;height: 44px;vertical-align: middle\">Motorcycles<\/td>\n<td style=\"width: 26.0352%;text-align: center;height: 44px;vertical-align: middle\">Cars + Motorcycles<\/td>\n<\/tr>\n<tr style=\"height: 44px\">\n<td style=\"width: 28.7736%;text-align: center;height: 44px;vertical-align: middle\">Sighthound (2018)<\/td>\n<td style=\"width: 21.2264%;text-align: center;height: 44px;vertical-align: middle\">58.4%<\/td>\n<td style=\"width: 25%;text-align: center;height: 44px;vertical-align: middle\">3.3%<\/td>\n<td style=\"width: 26.0352%;text-align: center;height: 44px;vertical-align: middle\">47.4%<\/td>\n<\/tr>\n<tr style=\"height: 44px\">\n<td style=\"width: 28.7736%;text-align: center;height: 44px;vertical-align: middle\">OpenALPR (2018)<\/td>\n<td style=\"width: 21.2264%;text-align: center;height: 44px;vertical-align: middle\">58.0%<\/td>\n<td style=\"width: 25%;text-align: center;height: 44px;vertical-align: middle\">22.8%<\/td>\n<td style=\"width: 26.0352%;text-align: center;height: 44px;vertical-align: middle\">50.9%<\/td>\n<\/tr>\n<tr style=\"height: 44px\">\n<td style=\"width: 28.7736%;text-align: center;height: 44px;vertical-align: middle\">Proposed (2018)<\/td>\n<td style=\"width: 21.2264%;text-align: center;height: 44px;vertical-align: middle\">72.2%<\/td>\n<td style=\"width: 25%;text-align: center;height: 44px;vertical-align: middle\">35.6%<\/td>\n<td style=\"width: 26.0352%;text-align: center;height: 44px;vertical-align: middle\">64.9%<\/td>\n<\/tr>\n<tr>\n<td style=\"width: 28.7736%;text-align: center;height: 44px;vertical-align: middle\"><a href=\"https:\/\/web.inf.ufpr.br\/vri\/publications\/layout-independent-alpr\/\" target=\"_blank\" rel=\"noopener noreferrer\"><strong>Proposed-Extended<\/strong><\/a><\/td>\n<td style=\"width: 21.2264%;text-align: center;vertical-align: middle\"><strong>95.9%<\/strong><\/td>\n<td style=\"width: 25%;text-align: center;vertical-align: middle\"><strong>66.3%<\/strong><\/td>\n<td style=\"width: 26.0352%;text-align: center;vertical-align: middle\"><strong>90.0%<\/strong><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p>\u00a0<\/p>\n<table style=\"border-collapse: collapse;width: 95%;height: 220px;text-align: center;margin-left: auto;margin-right: auto;margin-top: -35px\">\n<tbody>\n<tr style=\"height: 44px\">\n<td style=\"width: 28.7736%;text-align: center;height: 44px;vertical-align: middle\">ALPR System <strong>(with redundancy)<\/strong><\/td>\n<td style=\"width: 21.2264%;text-align: center;height: 44px;vertical-align: middle\">Cars<\/td>\n<td style=\"width: 25%;text-align: center;height: 44px;vertical-align: middle\">Motorcycles<\/td>\n<td style=\"width: 25.1842%;text-align: center;height: 44px;vertical-align: middle\">Cars + Motorcycles<\/td>\n<\/tr>\n<tr style=\"height: 44px\">\n<td style=\"width: 28.7736%;text-align: center;height: 44px;vertical-align: middle\">Sighthound (2018)<\/td>\n<td style=\"width: 21.2264%;text-align: center;height: 44px;vertical-align: middle\">70.8%<\/td>\n<td style=\"width: 25%;text-align: center;height: 44px;vertical-align: middle\">0.0%<\/td>\n<td style=\"width: 25.1842%;text-align: center;height: 44px;vertical-align: middle\">56.7%<\/td>\n<\/tr>\n<tr style=\"height: 44px\">\n<td style=\"width: 28.7736%;text-align: center;vertical-align: middle;height: 44px\">OpenALPR (2018)<\/td>\n<td style=\"width: 21.2264%;text-align: center;vertical-align: middle;height: 44px\">89.6%<\/td>\n<td style=\"width: 25%;text-align: center;vertical-align: middle;height: 44px\">0.0%<\/td>\n<td style=\"width: 25.1842%;text-align: center;vertical-align: middle;height: 44px\">71.7%<\/td>\n<\/tr>\n<tr style=\"height: 44px\">\n<td style=\"width: 28.7736%;text-align: center;height: 44px;vertical-align: middle\">Proposed (2018)<\/td>\n<td style=\"width: 21.2264%;text-align: center;vertical-align: middle\">83.3%<\/td>\n<td style=\"width: 25%;text-align: center;vertical-align: middle\">58.3%<\/td>\n<td style=\"width: 25.1842%;text-align: center;vertical-align: middle\">78.3%<\/td>\n<\/tr>\n<tr style=\"height: 44px\">\n<td style=\"width: 28.7736%;text-align: center;height: 44px;vertical-align: middle\"><a href=\"https:\/\/web.inf.ufpr.br\/vri\/publications\/layout-independent-alpr\/\" target=\"_blank\" rel=\"noopener noreferrer\"><strong>Proposed-Extended<\/strong><\/a><\/td>\n<td style=\"width: 21.2264%;text-align: center;vertical-align: middle;height: 44px\"><strong>98.3%<\/strong><\/td>\n<td style=\"width: 25%;text-align: center;vertical-align: middle;height: 44px\"><strong>70.0%<\/strong><\/td>\n<td style=\"width: 25.1842%;text-align: center;vertical-align: middle;height: 44px\"><strong>92.7%<\/strong><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n\n\n<h3 class=\"wp-block-heading\" id=\"contact\"><b>4. Related Publications&nbsp;<\/b><\/h3>\n\n\n\n<p class=\"indent\">A list of all papers on ALPR published by us can be seen\u00a0<a href=\"https:\/\/scholar.google.com\/scholar?hl=pt-BR&amp;as_sdt=0%2C5&amp;as_ylo=2018&amp;q=allintitle%3A+plate+OR+license+OR+vehicle+author%3A%22David+Menotti%22&amp;btnG=\" target=\"_blank\" rel=\"noreferrer noopener\"><strong>here<\/strong><\/a>.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\" id=\"contact\"><b>5. Contact&nbsp;<\/b><\/h3>\n\n\n\n<p class=\"indent\">Please contact the first author (<a rel=\"noreferrer noopener\" href=\"mailto:rblsantos@inf.ufpr.br\" target=\"_blank\">rblsantos@inf.ufpr.br<\/a>) with questions or comments.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>1. Paper Information 1.1. Authors Rayson Laroca, Evair Severo, Luiz A. Zanlorensi, Luiz S. Oliveira, Gabriel R. Gon\u00e7alves, William Robson Schwartz, David Menotti. Automatic License Plate Recognition (ALPR) has been a frequent topic of research due to many practical applications. <a href=\"https:\/\/web.inf.ufpr.br\/vri\/publications\/laroca2018robust\/\" class=\"read-more\">Read More &#8230;<\/a><\/p>\n","protected":false},"author":55,"featured_media":0,"parent":1252,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"footnotes":""},"class_list":["post-1175","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/web.inf.ufpr.br\/vri\/wp-json\/wp\/v2\/pages\/1175","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/web.inf.ufpr.br\/vri\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/web.inf.ufpr.br\/vri\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/web.inf.ufpr.br\/vri\/wp-json\/wp\/v2\/users\/55"}],"replies":[{"embeddable":true,"href":"https:\/\/web.inf.ufpr.br\/vri\/wp-json\/wp\/v2\/comments?post=1175"}],"version-history":[{"count":50,"href":"https:\/\/web.inf.ufpr.br\/vri\/wp-json\/wp\/v2\/pages\/1175\/revisions"}],"predecessor-version":[{"id":1982,"href":"https:\/\/web.inf.ufpr.br\/vri\/wp-json\/wp\/v2\/pages\/1175\/revisions\/1982"}],"up":[{"embeddable":true,"href":"https:\/\/web.inf.ufpr.br\/vri\/wp-json\/wp\/v2\/pages\/1252"}],"wp:attachment":[{"href":"https:\/\/web.inf.ufpr.br\/vri\/wp-json\/wp\/v2\/media?parent=1175"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}