Forest Inventory, LiDAR, and Patents

Forest Inventory, LiDAR, Forest Inventory, LiDAR, and Patentsand Patents

Jim FlewellingJim Flewelling

Presented at Silvilaser 2009,

College Station, TX, Oct. 14-16, 2009

Seattle Biometrics and Analysis, LLCflewelling seattlebiometrics.com

Minor corrections made November 2, 2009

IF YOU WANT TO OBTAIN A PATENT

PAY ATTENTION !

IF YOU WANT TO PREVENT A PATENT

PAY ATTENTION !

IF YOU WANT TO BE SUED FOR INFRINGEMENT

TAKE YOUR MORNING SIESTA !

THANK THIS MAN FOR NOT PATENTING “FOREST INVENTORY BY LASER”

ROSS NELSONTerje Gobakken

Dedicated to an InventorDedicated to an Inventor

Walter Bitterlich

(1908-2008)

8080thth, in 1988, 3 Relascopes, in 1988, 3 Relascopes

Photo by Kim Iles

Bitterlich – several inventionsBitterlich – several inventions

1. Angle-count Sampling

G = k × (Tree Count)

not patented

2. Angle

Gauge

3. Relascope, w Benno Hesske4. Model for critical height sampling and growth.

Theories from

Kitamura,

and Iles.

OUTLINEOUTLINE

• Patent Basics

• Disclaimer

• Inventory Methods Patents

• Ways to Influence Patents

• Patent Infringement

• Resources

Patent BasicsPatent Basics

• Patents exclude others from using the patented product or method.

• Patents have one or more claims.

• Each claim is a “mini patent”.

• Each claim has one or more elements.


• Patent Applications are– Submitted by all the true inventors.– Novel.– Unobvious.– Useful.– Of a patentable nature.

• Priority date: usually date the patent application is made.


• Prior art are publications or public disclosures before the priority date.

Patent Basics / ObligationsPatent Basics / Obligations

• Duties apply to inventors and others.

• Continuing duty of candor: to disclose all known material related to patentability.

(Slideshow by Moatz, 2007)

: http://www.ipo.org/AM/Template.cfm?Section=Calendar&Template=/CM/ContentDisplay.cfm&ContentID=15913

DisclaimerDisclaimer

• Nothing shown here is intended as a criticism of the inventors.

• Potential problems of the patents are raised.

• Speaker has no legal background.

• For your issues, get a good IP attorney.

OUTLINEOUTLINE

• Patent Basics

• Disclaimer




• Resources

Inventory Methods PatentsInventory Methods Patents

Inventors Priority Date

Key word in title Application # or US Patent

Assignee

Hyyppä 1999.10.28 Stand Attributes

(ALS)

Patent

6792684

Diware Oy

Rousselle et al 2002.05.03 Feature identification

(photo)

Patent

7212670

1st: Falcon, now

ImageTree Corp.

Kelle et al 2006.06.19 Feature Identification

(ALS)

Prov. Appl 60/814715

ImageTree Corp.

Kelle et al 2006.06.19 Probabilistic Sampling

(ALS)

Application

11/728099

ImageTree Corp.

Bortolot & McTague

2006.08.16 Forest Inventory (tree clusters, ALS)

Application

11/505189

Abandoned.

Hyyppä and Yu 2003.03.31 Growth of Trees

(ALS)

Application

10/803966

Geodeettinen Laitos

Savolainen et al 2004.03.15 Stand Attributes

(photo + lidar)

Application

10/598322

Blom Karita Oy,

Geodeettinen L..

Jupp et al 2001.02.09 Lidar System

(ALS + Terrestrial)

Patent

7187452

CSIRO

(Australia)

Welty et al 2007.06.22 Vegetation Attributes Patent

7474964

Weyerhaeuser Company

Hyyppä, J. Hyyppä, J. Priority Date 1999.10.08 Priority Date 1999.10.08

Title Method for Determination of Stand Attributes and a Computer Program to Perform the Method

Literature Hyyppä, J, Hyyppä, H and A. Samberg (1999, prior).

Hyyppä, J, and M. Inkinen (1999, subsequent)Hyyppä et al. (2001)

Hyyppä, J. Priority Date 1999.10.08 Hyyppä, J. Priority Date 1999.10.08 First of 20 ClaimsFirst of 20 Claims

Method for stand estimation.

Steps 1 and 2: obtain ALS data.

Step 3: Make a crown surface model or anything similar.

Step 4: Make any sort of stand estimates by building from single trees or groups of trees.

• “Method for determination of stand attributes with an instrument above the stand, comprising:

– moving a laser scanner in a first direction,

– collecting three-dimensional information from a stand by sweeping a laser beam of the laser scanner in a second direction substantially perpendicular to the first direction and by using such a number of sample hits that individual trees or groups of are identified,

– producing a three-dimensional presentation of a stand height from the collected three-dimensional information, and

– determining stand attributes, being characteristics of individual trees or groups of trees and characteristics for larger stand areas, from the three-dimensional presentation”

Hyyppä, J. Priority Date 1999.10.28 Hyyppä, J. Priority Date 1999.10.28 Potential Problem – Potential Problem – Not Novel and UnobviousNot Novel and Unobvious

• Nilsson (1996): “ If airborne laser data could be given both height and planimetric coordinates with high accuracy, single trees or groups of trees could be identified and detailed terrain models could be generated. this would be of great importance in many different situations, ...”

• Borgefors, G., T. Brandtberg, and F. Walter, (1999) delineated tree crowns from ALS data and noted that heights were known for the pixels within each crown and could be used to estimate tree heights.

• Brandtberg’s dissertation posted 1999.10.28 (too late!)• St-Onge published in November, 1999, but 1997 grant proposal states:

“The height of individual trees will be obtained from a local maximum technique also based on model-based segmentation in order to locate tree tops.”

• Pre-October 1999 grant proposals by Hyyppä, Borgefors, Andersen, Young Evans and Parker , Popescu, Koch and Friedlander ??????

Hyyppä, J. Priority Date 1999.10.08 Hyyppä, J. Priority Date 1999.10.08 Patent Family Patent Family

From patentlens.net Initiative of Cambia (AU), free.

Austria

Australia

Canada

Germany

Europe

Spain

Finland

Norway

U.S.

World

RousselleRousselle, A., Leppanen, V., McCrystal, D., Kelle, , A., Leppanen, V., McCrystal, D., Kelle, O. and R. Pliszka Priority Date 2002.05.03O. and R. Pliszka Priority Date 2002.05.03

• “Method of Feature Analysis”

• Literature: McCrystal (2003)

• Crown delineation based on photo pixels.

• Manual training procedure.

• Thirty-nine claims. Many steps in each.

• “Data model” to predict tree size based on crown size.

Rousselle, et alRousselle, et al Priority Date 2002.05.03Priority Date 2002.05.03Technological Limitation – Inappropriate ModelsTechnological Limitation – Inappropriate Models..

• “Data model” is a regression model fit to subjectively selected trees, using ground-measured crown sizes.

• Hyyppa’s “generally known formulas” to determine diameters are similar.

• Sampling theory is not being used.

• “.. Jim helped us understand that our methods of compiling this precise data leave it biased and unreliable” Adam Rousselle, personal communication. August, 2005.

Rousselle, et alRousselle, et al Priority Date 2002.05.03Priority Date 2002.05.03Example of change in inventorshipExample of change in inventorship..

• Original Application filed in May, 2002.• In December, 2004, the USPTO is petitioned to

name Robert Pliszka as an inventor. (Granted).• All must agree that he is a true inventor, and that

his original omission was not due to deceptive intent on their parts.

Note: a patent knowingly filed with incorrect inventorship is void ab initio.

Kelle, O.,Kelle, O., Macom, E., Pliszka, R., and N. Mathawan Macom, E., Pliszka, R., and N. Mathawan Priority Date 2006.06.19Priority Date 2006.06.19 Provisional ApplicationProvisional Application

Title Method of Feature Identification and Analysis

Law Provisional Applications do not become patents. They merely establish the subject matter, inventorship, and priority date.

Why Provisional applications are filed when an inventor is not yet ready to fully describe what it is he wants to claim. He may be concerned that someone else will invent or publish on the same method.

Kelle, O., Macom, E., Pliszka, R., and N. MathawanKelle, O., Macom, E., Pliszka, R., and N. Mathawan Priority Date 2006.06.19 Priority Date 2006.06.19 Provisional ApplicationProvisional Application

“Lidar and multispectral imagery are being used separately or together to identify and characterize individual tree crowns. Operationally it is feasible to collect data for entire forests, and to construct digital maps with stand boundaries and with polygons representing tree crown. Techniques to create crown polygons, assign species and impute tree characteristics are the subject of many ongoing research efforts in Scandinavia, the U.S. and elsewhere.

A starting point for sampling can be the map with the crown polygons attributed with species and possibly LIDAR height; this is used as a sampling frame for a statistically valid sampling inventory. The sample design ….”

“Description of the method.”

Kelle, O., Macom, E., Pliszka, R., and N. MathawanKelle, O., Macom, E., Pliszka, R., and N. Mathawan Priority Date 2006.06.19 Priority Date 2006.06.19 Regular ApplicationRegular Application

Title Remote Sensing and Probabalistic Sampling Based Forest Inventory Method.

Date Filed on March 23, 2007

What This application claims the priority of the earlier provisional application. The subject matter is little changed. Now claims are stated.

Literature Flewelling (2006)

Flewelling (2008) at Silvilaser 2008

Flewelling (2009)

Kelle, O., Macom, E., Pliszka, R., and N. MathawanKelle, O., Macom, E., Pliszka, R., and N. Mathawan Priority Date 2006.06.19 Priority Date 2006.06.19 First ClaimFirst Claim, Regular Appl., Regular Appl.

A remote sensing and probabilistic sampling based forest inventory method comprising:

a. processing remote sensing data indicative of tree attribute information for said forest, said remote sensing data comprising at least one of LiDAR data and digital images;

b. defining a sampling frame within said remote sensing data;

c. determining a field plot corresponding to said sampling frame and collecting field plot data therefrom, said field plot data comprising actual tree attribute information;

d. generating a correlated model by combining said field plot data with said remote sensing data corresponding to said sample frame; and

e. applying said correlated model to all said remote sensing data to produce a probabilistic forest inventory.

Compare with “Description of Method” in provisional application.

Kelle, et al. Priority Date 2006.06.19 Kelle, et al. Priority Date 2006.06.19 Potential Problem – Not novel.Potential Problem – Not novel.

• June 19, 2006 presentation by Flewelling covers similar material. (Not prior art).

• Abstract distributed on June 18 may be prior art.

Named inventors may publish up to one year before a U.S. patent application. This may preclude getting a European patent.

Details of an application. Details of an application. USPTO.govUSPTO.gov

• Enter the eBusiness tab / Public PAIR• Enter application number or publication

number or patent number.• Bibliographic data includes current

status.• Transaction history shows what has

happened.• Documents easy to download. (except

NPL)

eBusiness. Status & View Documents (Public Pair)

CLICK

HERE

Kelle, O., Macom, E., Pliszka, R., and N. MathawanKelle, O., Macom, E., Pliszka, R., and N. Mathawan STATUS, HOT OFF THE INTERNETSTATUS, HOT OFF THE INTERNET

• Thirty-five claims applied for.

• Most are rejected; Six may be OK.

• Attorney cancels 32 claims.

• Examiner approves patent for issuance with three claims.

• Another named inventor is now shown.

Inventory Methods Patents – Inventory Methods Patents – Next FiveNext Five

Inventors Priority Date

Key word in title Application # or US Patent

Assignee

Bortolot & McTague

2006.08.16 Forest Inventory (tree clusters, ALS)

Application

11/505189

Abandoned.

Hyyppä and Yu 2003.03.31 Growth of Trees

(ALS)

Application

10/803966

Geodeettinen Laitos

Savolainen et al 2004.03.15 Stand Attributes

(photo + lidar)

Application

10/598322

Blom Kartta Oy,

Geodeettinen

Jupp et al 2001.02.09 Lidar System

(ALS + Terrestrial)

Patent

7187452

CSIRO

(Australia)

Welty et al 2007.06.22 Vegetation Attributes Patent

7474964

Weyerhaeuser Company

There are many reasons that patents are applied for. Sometimes this is useful even if the plan is to abandon them.

OUTLINEOUTLINE

• Patent Basics

• Disclaimer




• Resources

Ways to Influence PatentsWays to Influence Patents• Defensive Publishing (*)

– Publish early.– SCRIBD.com – instant publications.– File your own patent applications.– “Foresee” a lot.

• Silvilaser proceedings.– SCIRUS.com can include.– Include in a regular publication series.– Petition USPTO to include in field of search.

* Adams et al. (2002)

Ways to Influence PatentsWays to Influence Patents

• Call attention to prior art.– Submit to patent office.

• Within 60 days of published appl. (U.S.)• After patent is issued.

– Submit to attorneys and inventors*• While patent is pending anywhere.

* See Cortina (2007)

OUTLINEOUTLINE

• Patent Basics

• Disclaimer




• Resources

Patent InfringementPatent Infringement

Research Use of Patented Methods *

Countries Permitted ?

EU, Norway, Japan, New Zealand, Korea, Mexico.

Probably

Australia, Canada Uncertain

United States Probably Not

* See Dent et al (2006). Do not rely on this table.

Patent InfringementPatent Infringement

• Your Defense– Not practicing the method.– Claims are invalid.– You have a license– Entire patent is invalid due to inequitable conduct.

• Willfull Infringement– Treble damages in the U.S.– Get an attorney’s written opinion first.– In the U.K., patent office may issue opinions.

OUTLINEOUTLINE

• Patent Basics

• Disclaimer




• Resources

ResourcesResources

WEB SITE Why

Uspto.gov Most info on pending patents

Patentstorm.com Easy, copies of patents

PatentlyO.com Legal forum

Gnu.org Public licensing

Gauss.ffii.org European patents

Groups.fsf.org Software issues

Resources / Resources / seattlebiometrics.comseattlebiometrics.com

• My site currently empty.

• U.S. patent publications by next week.

• May start with ftp.

• Would discussion forums be useful?

Inventorship and TeachingInventorship and Teaching

Walter Bitterlich and Kim Iles

“Courtesy and appreciation of the work of others are two of the reliable hallmarks of those in the top ranks of our profession”. Kim Iles (2003, p. 261).

These qualities are difficult to discern in many of today’s patent applications.

References not included in the ProceedingsReferences not included in the Proceedings

• Borgefors, G., T. Brandtberg, and F. Walter, 1999. Forest parameter extraction from airborne sensors. In; Proceedings of the ISPRS Conference Automatic Extraction of GIS Objects from Global Imagery (eds: Ebner, H., Eckstein, W., Heipke, C. and H. Mayer), ISPRS Conference, Sept. 8-10, 1999, Munich, Germany International Archives of Photogrammetry and Remote Sensing Vol. 32, Part 3-2W5, pp 151-158. (camera ready paper submitted May, 1999; proceedings distributed at the conference?)Brandtberg, T. 1999. Automated individual tree-based analysis of high spatial resolution remotely sensed data. Center for Image Analysis, Uppsala, Sweden. Thesis. [November, 1999, “nailed” on October 28.]

• Cortina, A. J. 2007. The duty to disclose: withhold at your patent’s peril. LocalTechwire (May 30, 2007). http://localtechwire.com/business/local_tech_wire/opinion/story/1454653/

• Iles, K. 2003. A Sampler of Inventory Topics. Kim Iles & Associates, Nanaimo, BC, Canada, 869 p. (ISBN 0-9732198-0-7. www.island.net/~kiles )

References not included in the ProceedingsReferences not included in the Proceedings

• HYYPPA , J., SCHARDT, M., HAGGRE´N, H., KOCH, B., LOHR, U., SCHERRER, H.U., PAANANEN, R., LUUKKONEN, H., ZIEGLER, M., HYYPPA¨ , H., PYYSALO, U., Friedlander, H., UUTTERA, J., WAGNER, S., INKINEN, M., WIMMER, A.,KUKKO, A., AHOKAS, E. and KARJALAINEN, M., 2001, HIGH-SCAN: The first European-wide attempt to derive single-tree information from laserscanner data. Photogrammetric Journal of Finland, 17, pp. 58–68.

• Koch B., and Friedlaender H., 1999, Erste Erfahrungen mit dem Einsatz von Laserscannerdaten zur Erfassung von vertikalen und horizontalen Strukturen im Wald, In Albertz J. (Hrsg.): Publikationen der Deutschen Gesellschaft für Photogrammetrie und Fernerkundung e.V., Band 8, pp. 335-343, Tagungsband der 19. Wissenschaftlich-Technischen Jahrestagung der DGPF, Essen, 13-15 October 1999. [I have not yet found a copy of this ]

• St-Onge, B.A. 1999. Estimating individual tree heights of the boreal forest using airborne laser altimetry and digital videography. La Jolla Conference, 9-11 November, 1999 (Commission III). p 174-184.

• Young, B., Evans, D.L., Parker, R.C. 2000. Methods for comparison of lidar and field measurements of loblolly pine. In proceedings: Second International Conference on Geospatial Information in Agriculture and Forestry, Lake Buena Vista, Florida, 10-12 January 2000: I-193-199. (CD distributed at the conference)