System and method for on-road detection of a vehicle using knowledge fusion

Number of patents in Portfolio can not be more than 2000

United States of America Patent

PATENT NO 7639841
APP PUB NO 20060177099A1
SERIAL NO

11311500

Stats

ATTORNEY / AGENT: (SPONSORED)

Importance

Loading Importance Indicators... loading....

Abstract

See full text

The present invention is directed to a system and method for on-road vehicle detection. A video sequence is received that is comprised of a plurality of image frames. A potential vehicle appearance is identified in an image frame. Known vehicle appearance information and scene geometry information are used to formulate initial hypotheses about vehicle appearance. The potential vehicle appearance is tracked over multiple successive image frames. Potential motion trajectories for the potential vehicle appearance are identified over the multiple image frames. Knowledge fusion of appearance, scene geometry and motion information models are applied to each image frame containing the trajectories. A confidence score is calculated for each trajectory. A trajectory with a high confidence score is determined to represent a vehicle appearance.

Loading the Abstract Image... loading....

First Claim

See full text

Family

Loading Family data... loading....

Patent Owner(s)

Patent OwnerAddressTotal Patents
SIEMENS CORPORATIONPRINCETON, NJ302
SIEMENS VDO AUTOMOTIVE AGHANNOVER23

International Classification(s)

Inventor(s)

Inventor Name Address # of filed Patents Total Citations
Comaniciu, Dorin Princeton Jct., US 269 3315
Köhler, Thorsten Deuerling, DE 8 252
Pellkofer, Martin Pentling, DE 8 461
Zhu, Ying Monmouth Junction, US 37 696

Cited Art Landscape

Patent Info (Count) # Cites Year
 
Other [Check patent profile for assignment information] (3)
* 5742699 Passive velocity measuring device 26 1995
* 6553130 Motor vehicle warning and control system and method 253 1996
* 6765480 Monocular computer vision aided road vehicle driving for safety 117 2002
 
INTELLECTUAL VENTURES FUND 83 LLC (1)
* 5600731 Method for temporally adaptive filtering of frames of a noisy image sequence using motion estimation 61 1991
 
Minnesota Mining and Manufacturing Company (3)
* 5434927 Method and apparatus for machine vision classification and tracking 106 1993
* 5621645 Automated lane definition for machine vision traffic detector 32 1995
* 5761326 Method and apparatus for machine vision classification and tracking 98 1995
 
SONY CORPORATION (1)
7386163 Obstacle recognition apparatus and method, obstacle recognition program, and mobile robot apparatus 35 2003
 
MEDIATEK INC. (1)
* 6643387 Apparatus and method for context-based indexing and retrieval of image sequences 99 2000
 
Autoliv Electronics (1)
2004/0252,863 Stereo-vision based imminent collision detection 24 2004
 
LOCKHEED MARTIN ELECTRONICS & MISSILES (1)
7274801 Automatic target recognition system with Elliptical Laplacian Pyramid filter 8 2003
 
SAMSUNG ELECTRONICS CO., LTD. (2)
7171023 Illumination-invariant object tracking method and image editing system using the same 91 2002
7352880 System and method for detecting and tracking a plurality of faces in real time by integrating visual ques 8 2003
 
HYUNDAI MOTOR COMPANY (1)
* 6795014 Method and apparatus for detecting vehicle distance 18 2003
 
Sony Electronics Inc. (1)
7596243 Extracting a moving object boundary 4 2005
 
HITACHI, LTD. (2)
* 5612686 Method and an apparatus for monitoring the environment around a vehicle and an operation support system using the same 74 1994
2006/0140,449 Apparatus and method for detecting vehicle 13 2005
 
TK HOLDINGS INC. (1)
7034742 Road curvature estimation and automotive target state estimation system 18 2003
 
AISIN SEIKI KABUSHIKI KAISHA (3)
* 4931937 Distance detector mounted on vehicle 81 1988
* 5555555 Apparatus which detects lines approximating an image by repeatedly narrowing an area of the image to be analyzed and increasing the resolution in the analyzed area 179 1994
* 5892855 Apparatus for detecting an object located ahead of a vehicle using plural cameras with different fields of view 96 1996
 
HONDA GIKEN KOGYO KABUSHIKI KAISHA (10)
* 6636257 Mobile body recognizing apparatus and motor vehicle monitoring apparatus 12 1999
* 6683969 Object detection system 161 2000
* 6731777 Object recognition system 15 2000
* 6853738 Optical object recognition system 42 2000
* 6327536 Vehicle environment monitoring system 111 2000
* 6531959 Position detecting device 30 2000
6963657 Object recognition system 34 2000
* 6590521 Object recognition system 46 2000
7221777 Image analysis device 4 2003
* 6847894 Vehicle control system, program and method 13 2003
 
OLYMPUS CORPORATION (1)
2005/0237,385 Stereo camera supporting apparatus, stereo camera supporting method, calibration detection apparatus, calibration correction apparatus, and stereo camera system 128 2005
 
AISIN AW CO., LTD. (1)
7519471 Driving support methods, apparatus, and programs 17 2005
 
MAZDA MOTOR CORPORATION (2)
* 6035053 Moving subject recognizing system for automotive vehicle 95 1997
* 6327522 Display apparatus for vehicle 44 2000
 
HONDA MOTOR CO., LTD. (2)
7463754 Adaptive probabilistic visual tracking with incremental subspace update 7 2004
7450736 Monocular tracking of 3D human motion with a coordinated mixture of factor analyzers 323 2006
 
NEVADA ASSET LIQUIDATORS, LLC (1)
* 5390133 Image processor for target detection and tracking 19 1992
 
KONINKLIJKE PHILIPS ELECTRONICS N.V. (3)
6973201 Person tagging in an image processing system utilizing a statistical model based on both appearance and geometric features 17 2000
* 6737963 Driver tailgating and following aid 67 2001
7190809 Enhanced background model employing object classification for improved background-foreground segmentation 13 2002
 
KABUSHIKI KAISHA TOSHIBA (8)
* 5777690 Device and method for detection of moving obstacles 31 1996
* 5991428 Moving object detection apparatus and method 30 1997
6963661 Obstacle detection system and method therefor 48 2000
7149327 Image processing apparatus and method 4 2003
2004/0057,599 Image processing apparatus and method 13 2003
6906620 Obstacle detection device and method therefor 91 2003
7132933 Obstacle detection device and method therefor 11 2005
2007/0211,917 Obstacle tracking apparatus and method 6 2006
 
NIPPON SOKEN, INC. (1)
* 6744380 Apparatus for monitoring area adjacent to vehicle 25 2002
 
LOCKHEED MARTIN CORPORATION (1)
* 6901152 Visual profile classification 2 2003
 
TOYOTA JIDOSHA KABUSHIKI KAISHA (3)
* 5638116 Object recognition apparatus and method 124 1994
* 5929785 Control device for vehicle 13 1997
* 6285393 Object recognition apparatus and method 203 1997
 
RAYTHEON COMPANY (2)
* 4739401 Target acquisition system and method 84 1985
* 6072889 Method and system for imaging target detection 58 1997
 
SIEMENS CORPORATION (3)
7336803 Method for scene modeling and change detection 12 2003
7248718 System and method for detecting a passing vehicle from dynamic background using robust information fusion 14 2005
7436980 Graphical object models for detection and tracking 3 2005
 
NORTHROP GRUMMAN SYSTEMS CORPORATION (1)
6937746 System and method for automatic recognition of formations in moving target indication data 6 2001
 
SIEMENS CORPORATION (REMAINING AS CO-OWNER) (2)
2004/0234,136 System and method for vehicle detection and tracking 44 2004
7466841 Method for traffic sign detection 20 2005
 
LUCAS INDUSTRIES LIMITED (1)
* 5765116 Driver assistance system for a vehicle 270 1996
 
TOY BIZ, INC. (1)
* 6263088 System and method for tracking movement of objects in a scene 164 1998
 
INTERNATIONAL BUSINESS MACHINES CORPORATION (2)
2001/0022,842 Method, apparatus and storage medium for adjusting the phase of sound from multiple speaker units 4 2001
2001/0022,842 Method, apparatus and storage medium for adjusting the phase of sound from multiple speaker units 4 2001
 
DAIMLER AG (1)
* 6556692 Image-processing method and apparatus for recognizing objects in traffic 13 1999
 
HEWLETT-PACKARD DEVELOPMENT COMPANY, L.P. (2)
* 6597816 Correcting distortion in an imaging system using parametric motion estimation 13 1998
* 6597801 Method for object registration via selection of models with dynamically ordered features 34 1999
 
SIEMENS AKTIENGESELLSCHAFT (1)
* 5592567 Method for detecting and separating the shadow of moving objects in a sequence of digital images 28 1995
 
VISTEON GLOBAL TECHNOLOGIES, INC. (2)
7561720 Single camera system and method for range and lateral position measurement of a preceding vehicle 6 2004
7561721 System and method for range measurement of a preceding vehicle 3 2005
 
DAIMLERCHRYSLER AG (1)
* 5937079 Method for stereo image object detection 83 1997
 
AGILENCE, INC. (3)
7088846 Video surveillance system that detects predefined behaviors based on predetermined patterns of movement through zones 35 2004
7127083 Video surveillance system with object detection and probability scoring based on object class 19 2004
2005/0104,962 Video surveillance system with rule-based reasoning and multiple-hypothesis scoring 7 2004
 
Kabushiki Kaisha Shinsangyokaihatsu (1)
* 4926346 Road image input system for vehicle control 51 1986
 
FUJITSU LIMITED (4)
* 6141435 Image processing apparatus 20 1996
* 6430303 Image processing apparatus 112 2000
2005/0169,501 Method and apparatus for determining driving lane of vehicle, and computer product 16 2004
2006/0171,563 Vehicle-mounted image processor 5 2006
 
KAMAN AEROSPACE CORPORATION (1)
* 5233541 Automatic target detection process 66 1990
 
HONDA ELESYS CO., LTD. (2)
* 6570998 Vehicle area detecting apparatus and vehicle area determining method 38 1999
* 6842531 Vehicle area detecting apparatus and vehicle area determining method 16 2003
 
SAMOFF CORPORATION (1)
2007/0086,621 Flexible layer tracking with weak online appearance model 30 2005
 
DENSO CORPORATION (2)
2005/0125,121 Vehicle driving assisting apparatus 24 2004
7042389 Device for detecting object in front of vehicle 24 2005
 
DELPHI TECHNOLOGIES, INC. (2)
* 4970653 Vision method of detecting lane boundaries and obstacles 377 1989
2006/0153,459 Object classification method for a collision warning system 9 2005
 
NATIONAL SECURITY AGENCY (1)
* 6298144 Device for and method of detecting motion in an image 74 1998
 
YAZAKI NORTH AMERICA, INC. (1)
7124027 Vehicular collision avoidance system 224 2002
 
SARNOFF CORPORATION (1)
* 6044166 Parallel-pipelined image processing system 37 1996
 
HTC CORPORATION (2)
* 6694044 Method for motion classification using switching linear dynamic system models 14 2000
6944317 Method for motion classification using switching linear dynamic systems models 9 2003
 
TEXAS INSTRUMENTS INCORPORATED (5)
* 4868871 Nonparametric imaging tracker 27 1987
* 5500904 System and method for indicating a change between images 50 1992
* 5969755 Motion based event detection system and method 326 1997
* 6628835 Method and system for defining and recognizing complex events in a video sequence 122 1999
* 6542621 Method of dealing with occlusion when tracking multiple objects and people in video sequences 60 1999
 
MITSUBISHI DENKI KABUSHIKI KAISHA (6)
* 5161632 Tracking control device for a vehicle 43 1991
* 5159557 Distance detecting apparatus and method for a vehicle 32 1991
* 5253050 Distance measuring device for measuring the distance between vehicles 32 1992
* 5369590 Inter-vehicle distance detecting device 51 1993
* 5530771 Image tracking device and image tracking method 57 1994
* 5617085 Method and apparatus for monitoring the surroundings of a vehicle and for detecting failure of the monitoring apparatus 245 1996
 
The United States of America as represented by the Secretary of the Navy (3)
* 5373456 Expert system for assessing accuracy of models of physical phenomena and for selecting alternate models in the presence of noise 32 1992
* 5537511 Neural network based data fusion system for source localization 13 1994
* 5937078 Target detection method from partial image of target 20 1996
 
COGNEX TECHNOLOGY AND INVESTMENT LLC (2)
2005/0093,697 Method and system for enhanced portal security through stereoscopy 19 2003
7397929 Method and apparatus for monitoring a passageway using 3D images 10 2003
 
Diablo Controls, Inc. (1)
7132959 Non-interfering vehicle detection 4 2004
 
YAZAKI CORPORATION (6)
* 5515448 Distance measuring apparatus of a target tracking type 323 1993
* 5521633 Motor vehicle obstacle monitoring system using optical flow processing 220 1993
* 6445809 Environment monitoring system 34 1999
* 6466684 Environment monitoring system 28 1999
* 6594583 Side-monitoring apparatus for motor vehicle 287 2001
7433496 In-vehicle image correcting device and night driving view field supporting device 4 2002
 
FUJITSU TEN LIMITED (1)
* 5555312 Automobile apparatus for road lane and vehicle ahead detection and ranging 164 1994
 
VIDIENT SYSTEMS, INC. (3)
2005/0104,959 Video surveillance system with trajectory hypothesis scoring based on at least one non-spatial parameter 7 2004
2005/0104,960 Video surveillance system with trajectory hypothesis spawning and local pruning 8 2004
7499571 Video surveillance system with rule-based reasoning and multiple-hypothesis scoring 6 2006
 
MAGNA ELECTRONICS INC. (1)
7038577 Object detection system for vehicle 384 2003
 
OMRON CORPORATION (1)
* 5910817 Object observing method and device 26 1996
 
SRI INTERNATIONAL (6)
* 6826292 Method and apparatus for tracking moving objects in a sequence of two-dimensional images using a dynamic layered representation 24 2000
* 6879705 Method and apparatus for tracking multiple objects in a video sequence 89 2000
7263209 Vehicular vision system 34 2003
7068815 Method and apparatus for ground detection and removal in vision systems 32 2004
2006/0165,277 Method and apparatus for unsupervised learning of discriminative edge measures for vehicle matching between non-overlapping cameras 19 2005
2008/0025,568 SYSTEM AND METHOD FOR DETECTING STILL OBJECTS IN IMAGES 15 2007
 
XEROX CORPORATION (1)
6954544 Visual motion analysis method for detecting arbitrary numbers of moving objects in image sequences 21 2002
 
MITSUBISHI ELECTRIC RESEARCH LABORATORIES, INC. (6)
* 6380934 Estimating targets using statistical properties of observations of known targets 24 1999
7224735 Adaptive background image updating 7 2003
7263472 Hidden markov model based object tracking and similarity metrics 14 2004
2005/0285,937 Unusual event detection in a video using object and frame features 9 2004
7526101 Tracking objects in videos with adaptive classifiers 36 2005
7466842 Modeling low frame rate videos with bayesian estimation 1 2005
 
PANASONIC INTELLECTUAL PROPERTY CORPORATION OF AMERICA (1)
6993159 Driving support system 25 2000
 
OL SECURITY LIMITED LIABILITY COMPANY (1)
7046822 Method of detecting objects within a wide range of a road vehicle 39 2000
 
FUJI JUKOGYO KABUSHIKI KAISHA (4)
* 5410346 System for monitoring condition outside vehicle using imaged picture by a plurality of television cameras 378 1993
* 5535144 Distance detection method and system using a stereoscopical imaging apparatus 193 1994
* 5530420 Running guide apparatus for vehicle capable of keeping safety at passing through narrow path and the method thereof 378 1994
* 6734787 Apparatus and method of recognizing vehicle travelling behind 15 2002
 
FLIR COMMERCIAL SYSTEMS, INC. (1)
* 6185314 System and method for matching image information to object model information 81 1998
 
SIEMENS MEDICAL SOLUTIONS USA, INC. (2)
* 6590999 Real-time tracking of non-rigid objects using mean shift 30 2000
7072494 Method and system for multi-modal component-based tracking of an object using robust information fusion 9 2005
 
NEC CORPORATION (1)
* 6549642 Method and system for detecting road area 4 1999
 
SUBARU CORPORATION (5)
* 6122597 Vehicle monitoring apparatus 185 1998
* 6370261 Vehicle surroundings monitoring apparatus 22 1999
* 6813370 Lane marker recognizing apparatus 63 2000
7356408 Information display apparatus and information display method 11 2004
2005/0143,887 Vehicle driving assist system 9 2004
 
MICROSOFT TECHNOLOGY LICENSING, LLC (3)
* 6757571 System and process for bootstrap initialization of vision-based tracking systems 37 2000
7035431 System and method for probabilistic exemplar-based pattern tracking 18 2002
7167578 Probabilistic exemplar-based pattern tracking 14 2005
 
HRL LABORATORIES, LLC (2)
* 6718259 Adaptive Kalman filter method for accurate estimation of forward path geometry of an automobile 17 2002
7409092 Method and apparatus for the surveillance of objects in images 41 2002
 
AVIGILON ANALYTICS CORPORATION (1)
2005/0002,572 Methods and systems for detecting objects of interest in spatio-temporal signals 47 2004
 
AVIGILON FORTRESS CORPORATION (1)
6999600 Video scene background maintenance using change detection and classification 68 2003
 
NATIONAL CHIAO TUNG UNIVERSITY (1)
7286707 Object-detection method multi-class Bhattacharyya Boost algorithm used therein 8 2005
 
HONEYWELL INTERNATIONAL INC. (3)
2003/0053,658 Surveillance system and methods regarding same 81 2001
2003/0053,659 Moving object assessment system and method 61 2001
2003/0123,703 Method for monitoring a moving object and system regarding same 63 2001
 
NISSAN MOTOR CO., LTD. (9)
* 6477260 Position measuring apparatus using a pair of electronic cameras 168 1999
6990216 Method and apparatus for estimating inter-vehicle distance using radar and camera 8 2001
* 6845172 Road lane marker recognition 11 2001
* 6879706 Apparatus and method for detecting traffic lane mark for automotive vehicle 19 2001
* 6873912 Vehicle tracking system 21 2003
7266454 Obstacle detection apparatus and method for automotive vehicle 2 2004
7542835 Vehicle image processing device 14 2004
2005/0102,070 Vehicle image processing device 14 2004
2006/0002,587 Image processing system and method for front-view image sensor 10 2005
 
HONEYWELL INC. (2)
* 4969036 System for computing the self-motion of moving images devices 68 1989
* 5036474 Motion detection and tracking from a mobile platform 27 1989
 
Strider Labs, Inc. (1)
2005/0286,767 System and method for 3D object recognition using range and intensity 64 2005
 
CONTINENTAL AUTOMOTIVE GMBH (1)
* 6301542 Cruise control system for vehicles, in particular for motor vehicles 26 2000
 
CONTINENTAL AUTOMOTIVE SYSTEMS, INC. (1)
* 6731204 Object detection system providing driver information through sound 17 2002
 
MOBILEYE VISION TECHNOLOGIES LTD. (1)
* 6704621 System and method for estimating ego-motion of a moving vehicle using successive images recorded along the vehicle's path of motion 154 2000
 
FORD GLOBAL TECHNOLOGIES, LLC (4)
2003/0055,563 Method for collision avoidance and collision mitigation 13 2002
* 6687577 Simple classification scheme for vehicle/pole/pedestrian detection 18 2002
7069130 Pre-crash sensing system and method for detecting and classifying objects 3 2003
7486802 Adaptive template object classification system with a template generator 25 2004
 
MATSUSHITA ELECTRIC INDUSTRIAL CO., LTD. (3)
* 5487116 Vehicle recognition apparatus 161 1994
* 5590217 Vehicle activity measuring apparatus 49 1994
2002/0134,151 Apparatus and method for measuring distances 67 2001
* Cited By Examiner

Patent Citation Ranking

Forward Cite Landscape

Patent Info (Count) # Cites Year
 
Other [Check patent profile for assignment information] (1)
* 7974475 Adaptive bayes image correlation 6 2009
 
INTERNATIONAL BUSINESS MACHINES CORPORATION (2)
* 8270675 Approach for resolving occlusions, splits and merges in video images 1 2011
* 2011/0317,920 APPROACH FOR RESOLVING OCCLUSIONS, SPLITS AND MERGES IN VIDEO IMAGES 0 2011
 
SIEMENS HEALTHCARE GMBH (1)
* 9569736 Intelligent medical image landmark detection 0 2016
 
TRW AUTOMOTIVE U.S. LLC (2)
* 8576115 Vehicle length sensors 24 2009
* 2011/0181,457 Vehicle Length Sensors 18 2009
 
MANDO CORPORATION (2)
* 9139226 Method, apparatus, and system for parking control 1 2012
* 2013/0085,636 METHOD, APPARATUS, AND SYSTEM FOR PARKING CONTROL 3 2012
 
IMAGE SENSING SYSTEMS, INC. (2)
8849554 Hybrid traffic system and associated method 6 2011
9472097 Roadway sensing systems 1 2014
 
ANALOG DEVICES GLOBAL (3)
* 9460354 Object detection 1 2013
* 2014/0133,698 OBJECT DETECTION 4 2013
9672431 Object detection 0 2013
 
INDUSTRIAL TECHNOLOGY RESEARCH INSTITUTE (1)
* 2012/0166,080 METHOD, SYSTEM AND COMPUTER-READABLE MEDIUM FOR RECONSTRUCTING MOVING PATH OF VEHICLE 6 2011
 
GM GLOBAL TECHNOLOGY OPERATIONS LLC (3)
* 8812226 Multiobject fusion module for collision preparation system 3 2010
* 2010/0191,391 MULTIOBJECT FUSION MODULE FOR COLLISION PREPARATION SYSTEM 35 2010
9250324 Probabilistic target selection and threat assessment method and application to intersection collision alert system 3 2013
 
Bendix Commercial Vehicle Systems LLC (1)
* 2015/0003,670 MEASURING THE RANGE TO AN OBJECT, IN AN IMAGE, USING SIZE CATEGORIZATION 1 2013
 
DENSO CORPORATION (2)
* 7920087 Apparatus for estimating state of vehicle located in frontward field 7 2008
* 2009/0040,095 Apparatus for estimating state of vehicle located in frontward field 6 2008
 
ASSA ABLOY AB (1)
8102799 Centralized wireless network for multi-room large properties 6 2007
 
TOYOTA MOTOR EUROPE NV/SA (1)
* 2012/0143,488 VEHICLE OR TRAFFIC CONTROL METHOD AND SYSTEM 9 2010
 
Pukoa Scientific, LLC (1)
8527445 Apparatus, system, and method for object detection and identification 4 2010
 
ROBERT BOSCH GMBH (1)
* 9747801 Method and device for determining surroundings 0 2013
 
FONDAZIONE BRUNO KESSLER (2)
* 8436913 Method for efficient target detection from images robust to occlusion 0 2010
* 2011/0050,940 Method for efficient target detection from images robust to occlusion 30 2010
 
BENDIX COMMERCIAL VEHICLE SYSTEMS LLC (1)
* 9058524 Measuring the range to an object, in an image, using size categorization 0 2013
 
TOYOTA JIDOSHA KABUSHIKI KAISHA (9)
* 2009/0010,482 Diagrammatizing Apparatus 5 2005
* 8989913 Travel route estimation device and travel route estimation method used in the same device 0 2008
* 2011/0264,302 TRAVEL ROUTE ESTIMATION DEVICE AND TRAVEL ROUTE ESTIMATION METHOD USED IN THE SAME DEVICE 2 2008
* 9053554 Object detection device using an image captured with an imaging unit carried on a movable body 1 2010
* 2012/0027,258 OBJECT DETECTION DEVICE 6 2010
* 8615109 Moving object trajectory estimating device 4 2011
* 8799201 Method and system for tracking objects 1 2011
* 2013/0031,045 METHOD AND SYSTEM FOR TRACKING OBJECTS 1 2011
* 9395448 Information indication apparatus 0 2015
 
WAYMO LLC (2)
* 8855849 Object detection based on known structures of an environment of an autonomous vehicle 2 2013
* 9026303 Object detection based on known structures of an environment of an autonomous vehicle 0 2014
 
CLARION CO., LTD. (1)
* 2013/0073,194 VEHICLE SYSTEMS, DEVICES, AND METHODS FOR RECOGNIZING EXTERNAL WORLDS 2 2012
 
CANON KABUSHIKI KAISHA (2)
* 8553086 Spatio-activity based mode matching 1 2009
* 2011/0043,699 SPATIO-ACTIVITY BASED MODE MATCHING 4 2009
* Cited By Examiner

Maintenance Fees

Fee Large entity fee small entity fee micro entity fee due date
7.5 Year Payment $3600.00 $1800.00 $900.00 Jun 29, 2017
11.5 Year Payment $7400.00 $3700.00 $1850.00 Jun 29, 2021
Fee Large entity fee small entity fee micro entity fee
Surcharge - 7.5 year - Late payment within 6 months $160.00 $80.00 $40.00
Surcharge - 11.5 year - Late payment within 6 months $160.00 $80.00 $40.00
Surcharge after expiration - Late payment is unavoidable $700.00 $350.00 $175.00
Surcharge after expiration - Late payment is unintentional $1,640.00 $820.00 $410.00