Vividata™'s OCR Shop XTR™/API Documentation v.5.1

Updated: November 26, 2003
Please contact Vividata support for archived copies of OCR Shop XTR/API documentation from previous releases.

---

Main Documentation

• Introduction
• About Optical Character Recognition
• Technical Specifications
• Installation
• API Design
• Basics of Usage
• Run-Time Usage
• Licensing
• Improving Recognition Accuracy
• Notes
• Contact Information

Additional Information

• Sample
• Frequently Asked Questions
• Glossary
  

---

Introduction

Main user documentation for the OCR Shop XTR™/API.

The OCR Shop XTR™/API Software Developer's Kit is the latest in Vividata's series of character recognition and image processing products. The OCR Shop XTR™/API provides unsurpassed recognition accuracy for Sun Solaris™ and Linux™ production environments and for developers looking to embed powerful OCR functionality into custom applications. Vividata™ has applied its dozen years of experience in document imaging, character recognition, and Unix technology, to combine the ScanSoft® SDK Version 5.0 with Vividata™'s extensive image-processing technologies to develop the OCR Shop XTR™/API. Further, with options to output PDF and HTML documents that combine recognized text with embedded images, OCR Shop XTR™/API provides the flexibility, accuracy and power to meet the production needs of high-end custom development requirements.

In this documentation, "you" refers to the application developer and "development seat" refers to the application developer's machine, where the OCR Shop XTR™/API is installed. The "end user" refers to the application end user and the "target system" refers to the application end user's machine.

Features and Benefits

Powerful

• Recognizes text in virtually any typeface and size, from 5 to 72 points.
• Supports 56 different languages including English, Eastern and Western European languages, Cyrillic-based languages (Russian), the Baltic languages, Turkish and Greek.

Accurate

• Automatically segments images to correctly recognize text on pages with complex or irregular layouts, including tables, reverse video, and line art.
• Increases recognition accuracy through built-in lexical processing and user-defined lexicons.

Flexible

• Decodes more than a dozen different image formats including TIFF, jpeg, gif, raw bitmap, PDF and PostScript®.
• Outputs text in standard ASCII, Unicode, XDOC and other formats, and image areas in up to 15 different formats.

About Optical Character Recognition

OCR Shop XTR™/API utilizes ScanSoft® OCR technology to accomplish its fast and accurate optical character recognition (OCR).

OCR is the process of transferring text from printed pages into a computer file on screen that can be edited without retyping. It is the process of translating bitmap image data into editable text. Text characters are designed by assigning a code corresponding to each key on the keyboard, be it a letter, number or symbol. There are a variety of different code sets in use, but the most common code set is the ASCII (American Standard Code for Information Interchange) table of character equivalents.

Technical Specifications

The OCR Shop XTR™/API is available for these platforms:

• Sun™ Solaris™ SPARC® (Solaris™ 2.7+)
• Linux™ x86 (Kernel 2.0 and higher)

The recognition engine is based on the ScanSoft® SDK 5.0

Note: All features are included with and are functional in the development seat. However, for end user run-time licenses, some features and input/output formats are supported as add-on options and are not licensed in a basic run-time license key. Please contact Vividata™, Inc. Sales and Support at (510) 658-6587 Ext. 107 for pricing information and details.

Supported input formats:

• GIF
• JPEG
• PBM
• PDF
• PNG
• PPM
• PostScript®
• Rasterfile
• SGI®-RGB
• TIFF
• XWD
• X11

Supported languages:

Text recognition uses a character set and one or more languages. English is the default language and all available languages are included with the development seat. For run-time licenses, additional languages are available as add-on language packs. Multiple languages may be specified for a single document, as long as they use the same character set.

Character sets:

• The character set defines the shape of the letters.
• Only one character set may be used at a time.
• English characters are an exception, and may be recognized in conjunction with any character set by using the special dm_english_chars value.
• The character set is chosen implicitely based on the language(s) selected by the user.

Languages:

• A language is associated with one character set.
• Only languages using the same character set may be specified concurrently.
• A document using a language with a dictionary is recognized based on the character shapes and with reference to the dictionary file specific to the chosen language.
• A document using a language without a dictionary is recognized based on the character shapes alone, without reference to a dictionary file.

• Languages with dictionaries:
  • czech (Central Europe - 1250)
  • danish (Latin 1 - 1252)
  • dutch (Latin 1 - 1252)
  • english (Latin 1 - 1252)
  • finnish (Latin 1 - 1252)
  • french (Latin 1 - 1252)
  • german (Latin 1 - 1252)
  • greek (Greek - 1253)
  • hungar (Hungarian; Central Europe - 1250)
  • italian (Latin 1 - 1252)
  • norsk (Norwegian; Latin 1 - 1252)
  • polish (Central Europe - 1250)
  • port (Portuguese; Latin 1 - 1252)
  • russian (Cyrillic -1251)
  • spanish (Latin 1 - 1252)
  • swedish (Latin 1 - 1252)
  • turkish (Turkish - 1254)

• Languages without dictionaries:
  • romanian (Central Europe - 1250)
  • estonian (Baltic - 1257)
  • afrikaans (Latin 1 - 1252)
  • albanian (Central Europe - 1250)
  • aymara (Latin 1 - 1252)
  • basque (Latin 1 - 1252)
  • breton (Latin 1 - 1252)
  • bulgarian (Cyrillic - 1251)
  • byelorussian (Cyrillic - 1251)
  • croatian (Central Europe - 1250)
  • faroese (Latin 1 - 1252)
  • flemish (Latin 1 - 1252)
  • friulian (Latin 1 - 1252)
  • gaelic (Latin 1 - 1252)
  • galician (Latin 1 - 1252)
  • greenlandic (Latin 1 - 1252)
  • hawaiian (Baltic - 1257)
  • icelandic (Latin 1 - 1252)
  • indonesian (Latin 1 - 1252)
  • kurdishlat (Kurdish Latin; Turkish - 1254)
  • latin (Latin 1 - 1252)
  • latvian (Baltic - 1257)
  • lithuanian (Baltic - 1257)
  • sorbianl (Sorbian - Lower; Central Europe - 1250)
  • macedonianc (Cyrillic - 1251)
  • malaysian (Latin 1 - 1252)
  • piginenglish (Latin 1 - 1252)
  • serbian (Cyrillic - 1251)
  • ukranian (Cyrillic - 1251)
  • catalan (Latin 1 - 1252)
  • sbcroatian (Serbo-Croation; Central Europe - 1250)
  • slovak (Central Europe - 1250)
  • slovenian (Central Europe - 1250)
  • swahili (Latin 1 - 1252)
  • tahitian (Latin 1 - 1252)
  • sorbianu (Sorbian - Upper; Central Europe - 1250)
  • welsh (Latin 1 - 1252)
  • frisianw (Frisian - West; Latin 1 - 1252)
  • zulu (Latin 1 - 1252)

Supported output text document formats:

• iso text, 8bit text, Unicode text, XDOC, pdf, html (to be supported in a future release)

Supported output subimage (graphical) formats:

• tiff, ras, epsf, x11, tiff-pack, tiff-g31d, tiff-g32d, tiff-g42d, pal-tiff, jpeg, png, xwd, rgb, rgb-rle, pdf
  
  
  

Installation

For installation of the OCR Shop XTR™/API, you should have downloaded one distribution file from Vividata's ftp site.

For Linux:

• xtrapi-linux-5.1r19

For Solaris:

• xtrapi-sun_5x-5.1r19

To install the API:

1. Place the distribution file in any temporary directory on your development seat machine.
2. Log in as root on your development seat machine.
3. Make sure the environment variable LD_LIBRARY_PATH is set. Normally this environment variable is set to at least /lib:/usr/lib.
4. From the directory containing the distribution file, run the command:

   For Linux:

   ./xtrapi-linux-5.01r0

   For Solaris:

   ./xtrapi-sun_5x-5.01r0

   
   

5. This will install the OCR Shop XTR™/API, placing these files on your system:

   /opt/Vividata/	All files in the API distribution
   /opt/Vividata/lib/	Libraries, resource files
   /opt/Vividata/include/	Headers
   /opt/Vividata/src/	Source code for a sample program
   /opt/Vividata/ghostscript/	Ghostscript resource files
   /opt/Vividata/bin/	Binaries

In addition, symbolic links are created in /usr/local/include/ and /usr/local/lib/ to the include files and libraries in /opt/Vividata/.

The default location for the development seat installation is /opt/Vividata/. If you decide to install the Vividata™ development seat elsewhere, you must set the environment variable VV_HOME before running the installer, installing the run-time license key, or using the OCR Shop XTR™/API functionality.

You can set the temporary directory used during installation and use of the OCR Shop XTR™/API by setting the environment variable TMP_DIR to a specific directory. /tmp is used by default.

You must install the run-time license key for testing included with your development seat before you can run a sample program. Please see the subsection Development Seat Licensing.

Please see the section on Basics of Usage for information on running the included sample program and using the API.

To uninstall:

To uninstall the API development seat, remove /opt/Vividata, as well as the symbolic links in /usr/local/include/:

• vvlicense
• vvocr
• vvutil

and the symbolic links in /usr/local/lib/:

• libboost_thread.a
• libvvocr_communicator.a
• libvvocr_factory.a
• libvvutil.a

API Design

The OCR Shop XTR™/API is designed to work as a client/server system, in which your application links statically with the provided communicator library, so that it may dynamically communicate with a daemon process containing the OCR engine. All function calls from your application are made to the statically linked libraries.

The main daemon process permits the creation of multiple instances of the OCR engine, serving one or more client programs. Furthermore, the API is designed in such a way that the OCR engine code is isolated from your application to increase process stability and to reduce dependencies, such that the OCR engine and your application do not share namespace, and a crash in your application will not affect the OCR engine.

Run-Time Usage

Run-Time usage describes the installation and use of your application on the end user's target system.

In order for the OCR engine to function, you must set up the licensing and resource files properly on the end user's target system.

• Run-Time Licensing
• Resource File Requirements
• Resource File Distributions
  

Run-Time Licensing

A run-time license key must be purchased for and installed on the target system in order for the OCR Shop XTR™/API software to run within your application.

Please see the section Licensing for information on how licensing works in general and specifically how to obtain and install purchased run-time licenses for an end user's target system.

Resource File Requirements

OCR Shop XTR™/API resource files must be installed on the target system in order for the OCR Shop XTR™/API software to run properly within your application.

You may install the resource files yourself in the process of installing your application, or we can provide a redistributable installer. The redistributable installer is similar to the development seat installer, but it installs only the appropriate resource files for a target machine. Please contact us for information on obtaining and using a redistributable installer.

The default location on the target system for the run-time resource files is /opt/Vividata/. If you decide to install the Vividata resource files elsewhere, you must set the environment variable VV_HOME before running the installer, installing the run-time license key, or using the OCR Shop XTR™/API functionality. This ensures that the OCR Shop XTR™/API code can find the resource files and license key file.

Resource File Distributions

Below is a list of the resource files you should distribute with your application. These files are all included with your development seat.

Note: You are not permitted to redistribute the OCR Shop XTR™/API libraries or include files.

• /opt/Vividata/bin/:
gs*
ocrxtrdaemon*
vvlmreread*
vvlmstop*

  
• /opt/Vividata/bin/linux/:
gs*
ocrxtrdaemon*
vvlmreread*
vvlmstop*

  
• /opt/Vividata/config/
The license key file will be installed here.
  
  
• /opt/Vividata/ghostscript/:
Fontmap
Fontmap.GS

  
• /opt/Vividata/lib/images/:
but-blank.gif
but-next.gif
but-prev.gif
index.gif

  
• /opt/Vividata/lib/langs/:
asciieng.lng
BALTIC.shp
CharSetTable.chr
CYRILLIC.shp
czech.lng
danish.lng
dutch.lng
english.lng
finnish.lng
french.lng
german.lng
greek.lng
GREEK.shp
hungar.lng
italian.lng
LATIN1.shp
LATIN2.shp
norsk.lng
polish.lng
port.lng
russian.lng
spanish.lng
swedish.lng
turkish.lng
TURKISH.shp

  Note: Only language files (.lng) and shape packs (.shp) that will be used on the target system, as regulated by the run-time licensing, are required to be installed on the target system.

Licensing

Licensing affects both the development seat and the end user's target system. A license key is typically tied to a machine's hardware id, and regulates how many concurrent instances (OCR engines) may run, PDF/PS input, PDF output, HTML output (in a future release), and languages. Alternative licensing is possible under contract on a case-by-case basis; please contact us for more information.

• Development Seat Licensing
• Run-Time Licensing
• How to Obtain Run-Time License Keys
• How to Install Run-Time License Keys
• Troubleshooting License Problems

Development Seat Licensing

One development seat consists of:

• One copy of the API distribution, including libraries, headers, and sample code to be installed on one machine.
  
  
• One point of contact, meaning one application developer working with the API who can contact Vividata™ for support in using the API.
  
  
• One run-time license key for testing on the development seat machine; this license key includes 5 licenses (instances), all features enabled, and all languages. You will receive the license key along with the API distribution.

Development seat licensing is regulated by the terms of the Vividata Developer License Agreement unless superceded by a previously executed contractual agreement.

How to obtain the run-time license key for testing which comes with your development seat:

A run-time license key with all features permitted will be emailed to you in conjunction with the development seat distribution.

How to install the development seat run-time license key for testing:

The license key comes encapuslated in a shell script. After installing the development seat distribution, you must install the run-time license key by running the shell script as root with the command: sh vvlicense.sh

This will place your license key in the file /opt/Vividata/config/vvlicense.dat.

If you are using a directory other than the default /opt/Vividata/ for the development seat installation directory, you must set the environment variable VV_HOME to the correct directory before installing the license key.

Run-time Licensing

In order to run your application on the end user's target system, a "run-time" license key must be installed. A license key defines how many licenses (instances) are available on a particular machine, as well as what features and languages are permitted for those licenses. All licenses on the same machine must share the same feature set.

A run-time license is checked out at the first occurrence of one of these events:

• The input image file is opened (vvEngAPI::vvOpenImageFile)
• The input image data is read into memory (vvEngAPI::vvReadImageData(const struct vvxtrImage * img))
• The output document is started (vvEngAPI::vvStartDoc)
• Preprocessing is run (vvEngAPI::vvPreprocess)
• Recognition is run (vvEngAPI::vvRecognize)

A run-time license is checked in when any one of these events occurs:

• The output document is ended (vvEngAPI::vvEndDoc)
• The OCR session is ended (vvEngAPI::vvEndOCRSes)
• The OCR engine is killed (vvEngAPI::vvKill)

A run-time license key is bound to the machine id (hostid on Solaris, or MAC addres on Linux) of the end user's target machine and controls these features:

• The number of licenses on the machine (one license is the same as one concurrent instance)
• PDF/PS input
• Language usage (corresponding to the dm_language value)
• PDF document output
• HTML output (supported in a future release)

The run-time license key should be installed on the target machine at the same time as your application. Under normal operating conditions, all licensing will work successfully if the license key file is in place when the application runs.

Please see the sections Run-Time Usage and How to Install a Run-Time License Key.

How to Obtain Run-Time License Keys

We offer an automated website to permit you to generate and download purchased run-time licenses for the end-users' target machines at your convenience. To obtain purchased run-time licenses through our website:

1. Go to http://www.vividata.com/account
2. Log in with your user id (usually your email address) and password (if you do not already have your user id or password, please contact us.)
3. Under Account Tools, create a new key by first selecting the correct invoice, then by clicking on "Create a license key for invoice".
4. Fill out the host information, number of instances to use in this license key, and any notes you would like to add. Note that 1 instance usually corresponds to 1 license; you may use multiple licenses on one machine by setting the number of instances.
5. On the next screen, review the license key information and click "Create Key" if there are no errors. Understand that by creating this key, you are using pre-purchased license(s), and the number of run-time licenses available will be decremented by the number of licenses you used for this key.
6. In the final step, your new license key will be displayed. Click on "Download this key" to download the shell script containing the key immediately. In the future, you may also access this key by clicking on "View all licenses" from the User Home page.

If you prefer, or if you have not already purchased run-time licenses and would like to do so, we are able to provide run-time licenses directly. Please contact Radcliffe Goddard at (510) 658-6587 Ext.107 or email support by filling out the web form at http://www.vividata.com/support_contact.html

How to Install a Run-Time License Key

A run-time license key must be installed on the end user's target machine before running your application.

1. You should have a license key file specifically generated for the target machine. The license key itself is encapsulated in a shell script.
2. Log in as root on the target machine.
3. If you are using a directory other than the default /opt/Vividata/ for Vividata™'s resource files and license key, you must set the environment variable VV_HOME to the correct directory.
4. Run the provided license key install file with the command:
   sh vvlicense.sh
5. The license key should now be installed on the target machine in /opt/Vividata/config/vvlicense.dat, or another directory as specified by VV_HOME.

Troubleshooting License Problems

• If the license manager enters a bad state, reset it by running the command:
  
  /opt/Vividata/bin/vvlmstop
  
  Then try running your application again.

• If vvlmstop does not seem to work, you can kill the license process by hand. On Linux, the process should be named "vvlicense" and on Solaris, it will be named "ocrxtrdaemon".

• If you installed the Vividata™ resource files and license key in a directory other than the default /opt/Vividata/, you must set the environment variable VV_HOME to the correct Vividata installation directory.

• License errors are expected if you attempt to use features that are not permitted by the license or if you try to check out more licenses (instances) than you are have.
  
  In both cases, you should check which features are permitted by the key and then verify that you are not trying to access an unlicensed feature.
  
  You can check which features should be allowed on a target machine by checking the license key installed on the machine:
  
  
  1. Open /opt/Vividata/config/vvlicense.dat to see the license key string itself.
  2. Pass this key string to the vvlmstatus utility:

     /opt/Vividata/bin/vvlmstatus -k <keystring>

  or by looking up the license key on our website:
  

  1. Go to http://www.vividata.com/account
  2. Log in with your user id and password (if you do not already have your user id or password, please contact us).
  3. From the User Home, click on the link "View all licenses".
  4. Find the correct license key in the list displayed. The features permitted by this key are listed along with the host information and number of instances.
     
     
  If you think there is a discrepancy between the features you purchased for the target machine and the features the key is permitting, please contact us.
  
  

• For any other questions regarding run-time license keys or licensing in general, please contact us.

Basics of Usage

The OCR Shop XTR™/API distribution consists of a number of include files and libraries.

Your application must include these headers:

• vvocr/vvxtrAPI.h
• vvocr/vvxtrFactory.h

Additional headers are included with the OCR Shop XTR™/API distribution for your reference and use.

Your application must link with these libraries:

• libvvocr_factory.a
• libvvocr_communicator.a
• libvvutil.a
• libboost_thread.a
• libpthread.a
• libsocket.a (for Solaris only)
• libposix4.a (for Solaris only)
• libstdc++.a

The first four of these libraries are included with the API distribution and are placed on your development seat system as part of the installation process; the other libraries you should already have as part of your operating system or as part of gcc.

The OCR Shop XTR™/API libraries are compiled with gcc 3.3, and we highly recommend using this compiler when linking with our libraries. Please let us know if you need to use a different compiler or encounter any problems.

Before you can run the sample program or your own application, the daemon must be started. To start the daemon, run the command:

• /opt/Vividata/bin/ocrxtrdaemon

A sample program is included with the OCR Shop XTR™/API distribution. To compile the sample program:

• Change to the /opt/Vividata/src/ directory.
• Run the command: gmake
• It will create the executable vvxtrSample.

You can run the sample program with the command:

• vvxtrSample localhost

It will prompt you for information to run a test document through the engine. We have included a sample input file: /opt/Vividata/src/letter.tif.

The daemon may be instructed to print out log information by setting the environment variable VV_DEBUG prior to starting the daemon. (See the include file vvLog.h for appropriate values for VV_DEBUG.) Similarily, your application can print out verbose log information; please see the call to vvLogSetLevel in vvxtrSample.cc and the FAQ question How do I generate log output?.

When you begin code development, we recommend you use the sample program vvxtrSample.cc, the header vvxtrAPI.h, and the webpages at http://www.vividata.com/xtrapi as your primary sources for documentation and information on the use of the API. The Frequently Asked Questions also provides valuable information. Please contact us for any additional questions.

Notes

The OCR Shop XTR™/API version 5.1 was released on August 19, 2003.

The API features:

• Input via a file or an image passed through memory
• Document output to ASCII, Unicode, XDOC, and PDF through a file or memory
• OCR input from GIF, JPEG, PBM, PDF, PNG, PPM, PostScript®, Rasterfile, SGI®-RGB, TIFF, XWD, and X11 image files
• Subimage output to a file or memory
• Preprocessing
• Recognition by page or region
• Various options for controlling preprocessing, recognition, input and output
• Control over region properties and boundaries, region creation and removal
• Licensing control over code compiled with our libraries

Major updates in this release include:

• Fully supported PDF output
• Many documentation updates, including the addition of a faq and more detailed information for the engine calls
• A more complicated sample program to better demonstrate usage of the API
• Various bug fixes in the API and the ScanSoft engine
• Smoother handling of vvxtrImages
• Access to the processed image data for subimage output is available through the dm_output_img_source options
• Addition of timeouts for vvRecognize and vvPreprocess
• Improved handling of vvKill
• Better handling if more than one ocrxtrdeamon is started

We plan future improvements to be included in subsequent releases, including:

• HTML output
• Detailed tweaking of deskewing parameters and image output options.
  
  
  

Contact Information

For support questions, please contact us:

• Online support form/Email: http://www.vividata.com/support_contact.html
  
  
• Phone: Radcliffe Goddard at (510) 658-6587 Ext.107
  
  

Vividata's website: http://www.vividata.com

Vividata™, Inc.
1300 66^th Street
Emeryville, CA 94608
U.S.A.

Phone: (510) 658-6587
Fax: (510) 658-6597
Toll-free Sales: (800) 704-2640

Documentation Copyright © 2003, Vividata™. All Rights Reserved.

---