Detectors recognize a format, not just a type
formex ships a library of detectors: small functions that answer one yes/no question about a column — “is each value written in format X?” Each is a pure predicate:
-
input: a character vector
x -
output: a logical vector the same length (
TRUEwhere the value matches,NAwherexisNA); detectors never error.
The word format is doing the work here. “Is this a date?” is too loose to be useful — almost any short string of digits and slashes could be a date. The detector that earns its place answers something tighter: is this a M/D/YYYY date, with the month first, slashes as separators, and real month and day numbers? That is a format style: a specific written shape, with its delimiters in specific places, drawn from a specific alphabet, often of a specific length, sometimes carrying a self-checking digit. A detector is good exactly when it pins that structure and rejects everything that merely resembles it.
Stress-testing the format with AutoType
So the hard part is not accepting real values — it is rejecting plausible-but-wrong ones that share the data type but break the format. To measure that, formex borrows the negative-example machinery from the AutoType framework (Yan & He, Auto-Type: Synthesizing Type-Detection Logic …, SIGMOD 2018), which synthesizes type detectors and the hard negatives to test them. Starting from a handful of real positives, it mutates them into near-misses along four tiers — each one aimed at a different part of the format’s structure:
| Tier | Name | What it changes | The structure it stresses |
|---|---|---|---|
| S1 | preserve-structure | replaces only alphanumeric characters, leaving the delimiters in place | the alphabet / values (for loose types it often regenerates a valid value, so it stresses little) |
| S2 | preserve-alphabet | replaces any character, but only with characters seen in real examples | the arrangement, while staying in-vocabulary |
| S3 | random | replaces characters with arbitrary printable ASCII | the alphabet itself (the easiest negatives) |
| SP | scramble-structure | keeps the alphanumerics and their order, but permutes where the punctuation/delimiters sit | the format style directly — a decimal point, a grouping comma, a symbol in the wrong position |
That last tier, SP, is the point of this guide’s framing: it holds the data type constant and attacks only the format style, catching a detector that checks “the right characters are present” but not “in the right places.” In that sense the fixtures test formats more than they test types.
The frozen mix of valid + labelled negatives is the bundled [data_type_tests] object. The report below applies each detector to it and reports pass rates per tier:
- Recall — share of real positives accepted. Should be 1.00.
-
S1 / S2 / S3 / SP — share of generated near-misses wrongly accepted (a false-positive rate). Lower is better; blank means that tier produced no distinct negatives. A high S1 is expected for loose formats (mutating the digits of
"$12.30"to"$47.99"is still a valid amount) — read S2/S3/SP for those.
What structure a detector leverages
Every detector in the catalogue names its strategy — the structure in the data it keys on. There are a handful of recurring kinds, from loosest to tightest:
- exact lookup — the value must be a member of a closed reference set (ISO currency codes, USPS states, NAICS codes). The structure is the list.
- small enum — a lookup against a short fixed vocabulary (census regions, NCES locales).
-
structured regex — a positional shape: delimiters and character classes in fixed places (
EIN= two digits, a dash, seven digits; aGUID= the 8-4-4-4-12 hex pattern). This is where the SP tier bites hardest. - regex + range — a shape plus a numeric bound, so the parts have to be real (an IPv4 octet is 0-255; an ISO date’s month is 1-12).
- anchor + length — a fixed width whose prefix must be valid (a county FIPS is five digits whose first two are a real state code).
- checksum — the value validates its own check digit (ISBN, ORCID, NPI, DEA), so a single wrong character is caught arithmetically. The tightest structure available.
- heuristic / composite — for loose real-world text (addresses), fires when several weaker components co-occur.
The catalogue below lists the detectors that ship with a frozen AutoType fixture, grouped by family, each with its one-line definition, the strategy (the structure it leverages), real examples, and the stress-test report. (A few newer detectors — is_currency_amount(), is_version(), is_boolean(), the phase and geometry detectors — are wired into the ontology but not yet in this frozen fixture, so they are not reported here.)
The catalogue
Currency
is_currency_code()
ISO 4217 three-letter alphabetic currency code (USD, EUR, JPY).
Precision: exact lookup
Strategy: Upper-cases the value and tests membership in the ISO 4217 code list.
Examples: AUD, CAD, CNY, DKK, EUR
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0.011 | 0.011 | 0.006 | NA |
Web / network
is_email()
An email address.
Precision: structured regex (loose)
Strategy: Regex: local part + @ + dotted domain + TLD; pragmatic, not full RFC 5322.
Examples: cjudgkins0@de.vu, csoutar1@bigcartel.com, afeavearyear2@skype.com, fmacgilrewy3@yellowpages.com, aeadmeades4@tamu.edu
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0.68 | 0.15 | 0.01 | 0.434 |
is_url()
An http/https/ftp URL.
Precision: structured regex
Strategy: Regex: scheme :// + dotted host + optional path/query.
Examples: http://ezproxy.baylor.edu/, https://www.ezproxy.unibo.it/, http://ezproxy.anzca.edu.au/, http://libproxy.aalto.fi/, http://elibrary.einstein.yu.edu/
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0.067 | 0.01 | 0 | 0.034 |
is_ipv4()
An IPv4 dotted-quad address.
Precision: regex + range
Strategy: Regex for four numeric octets, then validates each is 0-255.
Examples: 54.219.219.166, 49.200.88.156, 189.7.190.3, 206.120.6.24, 80.78.48.54
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0.421 | 0.077 | 0.003 | 0.06 |
is_ipv6()
An IPv6 address (full or ::-compressed).
Precision: structured regex
Strategy: Regex over eight hex groups plus common :: compressions.
Examples: 89a4:e654:d143:d1d0:b986:73e4:c8e1:4710, f385:9ac0:aede:bb7e:a620:b2a5:dd6b:ebfb, 4f:45b6:3366:d336:e41b:8df4:696:e02, 8316:886e:bebc:738:e0e0:215f:89de:89fb, 70eb:2fd0:84c2:9abd:7682:ff5b:afff:c24d
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 1 | 0.005 | 0 | 0 |
Color
is_hex_color()
A hex color such as #4d9cf8 or #fff.
Precision: structured regex
Strategy: Regex: # + 3 or 6 hex digits.
Examples: #4d9cf8, #1f3a8e, #eac3aa, #dc937b, #2cb0c3
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 1 | 0.44 | 0.024 | 0 |
is_rgb_color()
An RGB triple such as 220,220,220.
Precision: regex + range
Strategy: Three comma-separated integers, each validated 0-255.
Examples: 220,220,220, 255,250,250, 253,245,245, 240,240,240, 250,235,235
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0.315 | 0.07 | 0.004 | 0.035 |
Hash / UUID
is_md5()
An MD5 message digest.
Precision: fixed-length regex
Strategy: Exactly 32 hexadecimal characters.
Examples: 6e8a6448a62364225da9991fd8e1ef8c, 3df4ffb6eb24fffab3763041f13dbdea, b205e37bafd229efeb6345565ce428a0, d604c1cd26544a8c30f0bb20a689eac7, 793fce16a0a172e58c193b2899e0c025
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 1 | 1 | 0 | NA |
is_sha1()
A SHA-1 message digest.
Precision: fixed-length regex
Strategy: Exactly 40 hexadecimal characters.
Examples: 16de70e8fcd093964a448a86bc89b1c607152ead, efb6759998e36322258c172aa7e8ba6416b3235f, f1d48e18f51de36d01bafab5bfa68d16a8c0192b, 0ceee2d04e239856ecf18f93b5ee75d72e031f92, ac8a1226405cab9d6049304163d658c5ae611c1a
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 1 | 1 | 0 | NA |
is_sha256()
A SHA-256 message digest.
Precision: fixed-length regex
Strategy: Exactly 64 hexadecimal characters.
Examples: 96a162ea0e727b109724d95fa66659aacc2f5a1478c12bb779aa9c1fbd510450, 59d3b1aafad9d120e6df4a1a6ae6b5edf006be7a044e646b63a9feb8e909c900, 18f778c3525b95d98292595ebd8a0c5ac31bfb6cddeafefff04089468e304b96, e93042fecce04326c178d34b88c70f3ef8e90aaaefbc14e53a17a772c149b113, 21995f09908d3bb4f2c7eaf99a50cbad991aeb652208bb71c55e3161fcfaa493
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 1 | 1 | 0 | NA |
is_guid()
A GUID / UUID.
Precision: structured regex
Strategy: The 8-4-4-4-12 hexadecimal UUID pattern.
Examples: f2a59360-1d1a-49e8-9a2f-f9cdf0d5a811, 5ff3cc50-709c-4ba2-88a7-a2e4cca9f32c, 241c3e6c-1aa5-41c6-bbac-8ad08bdfcde9, 12e8b5a7-a1be-4dec-933a-6f9b040f8da3, 2eddc761-2258-4305-9e7b-c1087df3165c
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 1 | 0.035 | 0 | 0 |
Scientific / bio IDs
is_ensembl_id()
An Ensembl gene/transcript/protein ID (ENSG…, ENSMUSG…).
Precision: structured regex
Strategy: Regex: ENS + optional species + feature letter (E/G/T/P) + 11 digits.
Examples: ENSMUSG00000102659, ENSMUSG00000093178, ENST00000407559, ENSG00000139618, ENSG00000158636
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0 | 0 | 0 | NA |
is_uniprot_id()
A UniProtKB protein accession (P33667, Q9BWD1).
Precision: structured regex
Strategy: The official UniProt accession-number pattern.
Examples: P33667, P17888, P02338, Q9BWD1, Q46806
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0.311 | 0.311 | 0.017 | NA |
is_icd9_code()
An ICD-9-CM diagnosis code.
Precision: structured regex
Strategy: Regex: three digits (or an E/V code) + optional decimal detail.
Examples: 078.11, 008.8, 005.9, 117.9, 003.0
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0.652 | 0.294 | 0.013 | 0 |
is_icd10_code()
An ICD-10 code (I61, D76.0).
Precision: structured regex
Strategy: Regex: a letter + two digits + optional dotted detail.
Examples: I61, I63, G45, D76.0, D76.3
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0.399 | 0.242 | 0.036 | 0 |
is_snp_id()
A dbSNP reference SNP ID (rs3918290).
Precision: structured regex
Strategy: Regex: rs followed by digits.
Examples: rs3918290, rs1801159, rs1801265, rs17376848, rs1801160
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0.139 | 0.139 | 0 | NA |
is_asin()
An Amazon Standard Identification Number.
Precision: structured regex
Strategy: Regex: B0 followed by eight alphanumerics.
Examples: B01L7DYXL0, B0065RHC2Q, B004M8T1XS, B004M8T30O, B0065RRJGK
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0.262 | 0.262 | 0.013 | NA |
is_bibcode()
A 19-character NASA ADS bibliographic code.
Precision: structured regex
Strategy: Regex: 4-digit year + 14 code characters + a final letter.
Examples: 2004PASP..116...98K, 2003ApJ...597..957H, 2003A&A...405..175M, 2003ApJ...590..895W, 2003A&A...404..249B
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0.175 | 0.179 | 0.017 | 0.521 |
Codes / lookups
is_roman()
A Roman numeral.
Precision: structured regex
Strategy: The canonical well-formed Roman-numeral regex (rejects e.g. IIII).
Examples: I, II, III, IV, V
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0.248 | 0.248 | 0.01 | NA |
is_http_status()
An HTTP status code.
Precision: regex + range
Strategy: Three digits within 100-599.
Examples: 100, 101, 200, 201, 202
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0.701 | 0.701 | 0.015 | NA |
is_unix_time()
A Unix epoch timestamp in seconds.
Precision: regex + range (loose)
Strategy: 9-10 digits within a plausible epoch range; inherently loose.
Examples: 1513036980, 1503296497, 1451638800, 1360228214, 1497917283
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0.996 | 0.996 | 0.004 | NA |
is_zip_code()
A US ZIP or ZIP+4 code.
Precision: structured regex
Strategy: Five digits, optionally + four.
Examples: 98052, 15206, 32792, 50613, 27292
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 1 | 0.743 | 0.019 | 0 |
is_uk_postcode()
A UK postcode.
Precision: structured regex
Strategy: The standard UK postcode outward/inward pattern.
Examples: SA43 2BU, PO12 2DU, KA20 3LN, MK43 7TA, DT4 8US
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0.258 | 0.109 | 0.046 | 0 |
is_ca_postal_code()
A Canadian postal code or 3-character FSA.
Precision: structured regex
Strategy: Letter-digit-letter, optionally followed by digit-letter-digit.
Examples: V5K, K0H, T4A, T4B, T9S
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0.279 | 0.233 | 0.152 | 0 |
Date / time
is_iso_date()
An ISO 8601 calendar date (YYYY-MM-DD).
Precision: regex + range
Strategy: Regex + a month/day range check.
Examples: 2016-03-15, 2016-03-13, 2016-03-12, 2016-03-11, 2020-01-01
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0.337 | 0.085 | 0 | 0 |
is_iso_datetime()
An ISO 8601 datetime with optional fraction/zone.
Precision: structured regex
Strategy: Regex over date + T + time + optional .fraction and Z/offset.
Examples: 1987-02-06T06:00:37, 1988-01-12T06:34:58, 1930-07-06T02:08:45, 2009-06-15T13:45:30, 2010-06-15T13:45:30
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0.165 | 0 | 0 | 0 |
is_us_date()
A US numeric date (M/D/YY).
Precision: regex + range
Strategy: Regex + month/day validation.
Examples: 3/3/16, 1/27/16, 12/31/99, 7/4/2016, 2/29/2020
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0.615 | 0.125 | 0 | 0.032 |
is_us_datetime()
A US numeric datetime with AM/PM.
Precision: structured regex
Strategy: Regex: M/D/YYYY (slash or dash) + time + optional AM/PM.
Examples: 2/6/1987 6:00:37 AM, 1/12/1988 6:34:58 AM, 7/6/1930 2:08:45 AM, 7/15/1971 6:43:21 AM, 05-13-2009 12:35 PM
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0.085 | 0 | 0 | 0.01 |
is_abbr_month_date()
Day-month-year with an abbreviated month (1-Jul-14).
Precision: structured regex
Strategy: Regex: day + separator + 3-letter month + year.
Examples: 1-Jul-14, 2-Jul-14, 3-Jul-14, 4-Jul-14, 21/Nov/2012
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0.02 | 0.005 | 0 | 0 |
is_rfc2822_datetime()
An RFC 2822 / HTTP datetime.
Precision: structured regex
Strategy: Regex: weekday, DD Mon YYYY hh:mm:ss + zone.
Examples: Fri, 06 Feb 1987 06:00:37 GMT, Tue, 12 Jan 1988 06:34:58 GMT, Sun, 06 Jul 1930 02:08:45 GMT, Thu, 15 Jul 1971 06:43:21 GMT, Mon, 01 Jan 2001 00:00:00 GMT
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0 | 0 | 0 | 0 |
is_long_date()
A verbose English date (Friday, June 16, 1967).
Precision: structured regex
Strategy: Regex allowing optional weekday, full/abbrev month, day, year (with era) and time; requires a weekday or a year so it does not overlap Month-Day.
Examples: Friday, June 16, 1967, Thursday, June 9, 1955, Wednesday, May 17, 1967, Friday, June 16, 1967 8:46:44 AM, Friday, February 6, 1987 2:00:37 PM
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0 | 0 | 0 | 0 |
is_month_year()
A Month-Year value (February 1987).
Precision: structured regex
Strategy: Regex: month name + optional comma + 4-digit year.
Examples: February 1987, January 1988, July 1930, July 1971, August, 2005
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0 | 0 | 0 | 0 |
is_month_day()
A Month-Day value with no year (February 6).
Precision: structured regex
Strategy: Regex: month name + 1-2 digit day.
Examples: February 6, January 12, July 6, July 15, March 1
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0.005 | 0 | 0.005 | 0 |
is_clock_time()
A clock time (6:00:37 AM).
Precision: regex + range
Strategy: Regex + hour/minute range validation.
Examples: 6:00:37 AM, 6:34:58 AM, 2:08:45 AM, 6:43:21 AM, 11:59 PM
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0.115 | 0.02 | 0 | 0 |
Address
is_street_address()
A street-address line (2325 140th St).
Precision: heuristic regex
Strategy: Regex: house number + name + a street-type suffix (St/Ave/Blvd/…).
Examples: 2325 140th St, 303 Aspen Ct, 1225 S Walnut St, 30 Grandview Ct, 100 Main St
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0.013 | 0.013 | 0.003 | 0.047 |
is_po_box()
A PO box (PO Box 12, P.O.B. 3).
Precision: structured regex
Strategy: Regex covering PO / POB / Post Office Box variants + a number.
Examples: PO Box 1234, P.O. Box 56, POB 789, Post Office Box 42, PO BOX 100
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0.025 | 0.01 | 0 | 0.073 |
is_city_name()
A US city / place name.
Precision: exact lookup (partial coverage)
Strategy: Normalized membership test against ~1,000 larger US cities; smaller towns miss by design (precision over recall).
Examples: Chicago, New Orleans, Springfield, Pittsburgh, Bloomington
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0 | 0 | 0 | 0 |
is_city_state()
A ‘City, State’ value.
Precision: regex + lookup
Strategy: Splits on the comma and validates the trailing token is a real state (abbrev or name).
Examples: Bloomington, IN, Cheshire, CT, Bloomington, Indiana, Los Angeles, CA, New York, NY
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0.31 | 0.062 | 0.05 | 0.307 |
is_state_zip()
A ‘State ZIP’ value.
Precision: regex + lookup
Strategy: Regex whose leading token must be a real state, followed by a ZIP.
Examples: IN 47401, CT, 06410, Indiana 47401, CA 90210, TX 78701
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0.03 | 0.015 | 0 | 0 |
is_address()
A full postal address.
Precision: heuristic composite
Strategy: Scans one string for street / PO box / city / state / ZIP components and fires when at least three co-occur.
Examples: 1225 S Walnut St, Bloomington, IN 47401, 30 Grandview Ct, Cheshire, CT 06410, 1600 Pennsylvania Ave, Washington, DC 20500, 350 Fifth Avenue, New York, NY 10118, 742 Evergreen Terrace, Springfield, IL 62704
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0.005 | 0 | 0 | 0.07 |
Geography
is_state_fips()
A 2-digit state FIPS code.
Precision: exact lookup
Strategy: Membership in the valid state/territory FIPS set.
Examples: 06, 36, 48, 17, 12
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0.528 | 0.528 | 0.023 | NA |
is_county_fips()
A 5-digit county FIPS (GEOID).
Precision: anchor + length
Strategy: Five digits whose first two are a valid state FIPS.
Examples: 06037, 36061, 48201, 17031, 12086
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0.742 | 0.742 | 0.016 | NA |
is_tract_fips()
An 11-digit census-tract GEOID.
Precision: anchor + length
Strategy: Eleven digits with a valid state prefix.
Examples: 06037206200, 36061014500, 17031839100, 48201100000, 12086001000
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0.785 | 0.785 | 0 | NA |
is_block_group_fips()
A 12-digit census block-group GEOID.
Precision: anchor + length
Strategy: Twelve digits with a valid state prefix.
Examples: 060372062001, 360610145001, 170318391001, 482011000001, 120860010001
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0.75 | 0.75 | 0 | NA |
is_block_fips()
A 15-digit census-block GEOID.
Precision: anchor + length
Strategy: Fifteen digits with a valid state prefix.
Examples: 060372062001001, 360610145001000, 170318391001002, 482011000001003, 120860010001000
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0.731 | 0.731 | 0 | NA |
is_cbsa()
A 5-digit Core Based Statistical Area code.
Precision: exact lookup
Strategy: Membership in the Census CBSA code list.
Examples: 10100, 10140, 10180, 10220, 10300
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0.141 | 0.141 | 0.013 | NA |
is_csa()
A 3-digit Combined Statistical Area code.
Precision: exact lookup
Strategy: Membership in the Census CSA code list.
Examples: 101, 220, 364, 408, 348
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0.275 | 0.275 | 0.01 | NA |
is_metro_area_name()
A CBSA / CSA area title (metro-area name).
Precision: exact lookup (partial)
Strategy: Normalized membership against the official metro-area titles (full and state-stripped forms); distinct from city_name, which will not match these hyphenated multi-city titles.
Examples: Aberdeen, SD, Abilene, TX, Akron, OH, Athens-Clarke County, GA, Corpus Christi, TX
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0 | 0 | 0 | 0 |
is_nces_locale()
An NCES locale code (11-13, 21-23, 31-33, 41-43).
Precision: small enum
Strategy: Two-digit enum for City/Suburb/Town/Rural x Large/Midsize/Small.
Examples: 11, 12, 13, 21, 22
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0 | 0 | 0 | NA |
Government / entity IDs
is_ssn()
A US Social Security Number.
Precision: regex + rules
Strategy: ddd-dd-dddd with issuance validity rules (area not 000/666/900+, group not 00, serial not 0000).
Examples: 540-39-4268, 518-07-5084, 531-33-7396, 212-61-9091, 478-46-4157
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0.945 | 0.21 | 0 | 0 |
is_ein()
A US Employer Identification Number.
Precision: structured regex
Strategy: Two digits, a dash, seven digits.
Examples: 15-4464349, 58-9042990, 17-5287634, 11-9778467, 79-3293856
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 1 | 0.325 | 0 | 0 |
is_dea()
A DEA registration number.
Precision: checksum
Strategy: Two letters + seven digits, with the DEA check digit validated.
Examples: AP5836727, BJ6125341, AD0865937, AS9432042, AJ3247194
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0.018 | 0.018 | 0 | NA |
is_chinese_resident_id()
A Chinese resident identity-card number.
Precision: checksum
Strategy: Eighteen characters with an ISO 7064 mod-11-2 check character.
Examples: 141002197808194474, 431026197506218370, 210811198609188735, 371625197902109782, 140427197602187066
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0.093 | 0.093 | 0 | NA |
is_uei()
A SAM.gov Unique Entity Identifier (the 12-character code that replaced the DUNS number).
Precision: structured regex
Strategy: Twelve alphanumerics excluding the letters I and O, first character not 0.
Examples: ZQGGHJH74DW7, R7KZ8N2QP4M5, H3JD9FGT2WX6, AB1CD2EF3GH4, M5N6P7Q8R9S2
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 1 | 1 | 0.005 | NA |
Phone
is_phone()
A US / NANP phone number.
Precision: loose regex
Strategy: Regex over the common US formats and separators.
Examples: 510-577-3201, (309) 647-1820, (609)-993-3001, 650-384-9911, 1-650-384-9911
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0.875 | 0.021 | 0 | 0 |
Research / scholarly IDs
is_doi()
A Digital Object Identifier.
Precision: structured regex
Strategy: Regex: 10.
Examples: 10.1109/ICDCSW.2017.43, 10.1109/IRI.2016.84, 10.1109/ACC.2016.7526681, 10.1109/PIMRC.2015.7343424, 10.1109/ICICA.2014.47
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0.045 | 0.005 | 0 | 0.131 |
is_orcid()
An ORCID researcher iD.
Precision: checksum
Strategy: dddd-dddd-dddd-dddC with an ISO 7064 mod-11-2 check digit (C may be X).
Examples: 0000-0002-0488-8591, 0000-0002-6390-6569, 0000-0001-8666-7241, 0000-0001-6352-7991, 0000-0003-2738-1803
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0.058 | 0.013 | 0 | 0 |
is_isbn()
An ISBN-10 or ISBN-13.
Precision: checksum
Strategy: Validates the ISBN-10 (mod 11) or ISBN-13 (mod 10) check digit.
Examples: 140955525-9, 038027303-9, 489911182-7, 099337131-0, 712334370-0
| Recall | S1 | S2 | S3 | SP |
|---|---|---|---|---|
| 1 | 0.058 | 0.025 | 0 | 1 |
Adding a new detector
A detector is just a pure is_<type>() predicate, so adding one is self-contained. Collect 10-25 example values of the new type and build the same positive + hard-negative fixture the report above uses with [generate_autotype_test_cases()] (it generates the S1/S2/S3/SP negatives via [generate_autotype_negatives()]):
cases <- generate_autotype_test_cases(
P = c("ENSG00000139618", "ENSMUSG00000102659", "ENST00000407559"),
data_type = "ensembl_id")Write the is_ensembl_id() predicate, then score it against that fixture with [build_autotype_results()] and [generate_autotype_report()] to read the same recall/precision table shown here before you adopt it. Once you are happy:
- Drop the function into an
R/dt-<family>.Rfile (a character-in, logical-out predicate that never errors). - Add its positive examples to
data-raw/datatypes/<family>.R. - Register one line in
R/00-data-type-registry.Rsoguess_data_type()andscore_data_type()pick it up, and add a row to the ontology (inst/extdata/research_data_type_ontology_v7.csv) for its semantic type.
See ?generate_autotype_test_cases, ?build_autotype_results, and ?generate_autotype_report for details.