Programmatically determining the form of the English indefinite article

Question

Not sure if this is the correct place for this question. I am writing some code to place the correct indefinite article before a given noun. To do this I am looking at the first letter of the noun to decide whether to use 'a' or 'an'. I know this won't always give the correct output (e.g. An unicorn, A hour), as it is the sound rather than the actual letter that needs to be considered. To correct this I am looking for a list of all nouns where using the first letter will give the wrong result. Does anyone know where I can find such a list?

Answer 1

Even given a comprehensive list of such nouns, it won't be anything like sufficient. The reason is, of course, that the choice between a and an depends not on whether the head noun in the noun phrase begins with a vowel, but whether the following word begins with a vowel.

We can see this contrast in (1-4) below:

1. an elephant
2. a big elephant
3. a university
4. an unusual university

In (1) the head noun elephant begins with /e/ [or /ɛ/ if you prefer that symbol for the same phoneme]. Because the indefinite article occurs directly before this vowel, we see the form an. In (2) however, the article precedes the consonant /b/ in big and so we see the form a. We see the same kind of pattern in (3) and (4), where university begins with the consonant /j/, but unusual begins with the vowel /ʌ/.

You therefore need a list of all words spelled with consonant and pronounced with a vowel, or vice versa, that could occur after an indefinite article in English. In addition to nouns, this would have to include other determiners (e.g. numerals), prepositions, adjectives, adverbs, and verbs. It's probably safest to get a list of all such words in English regardless of their word category!

Answer 2

You need a pronouncing dictionary: the CMU dictionary is pretty easy to use. Using the web interface, you can try to get "univariant",which directs you to Logios, which may generate a satisfactory pronunciation my rule.

Answer 3

I think it would be easier to generate your own data because then you don't need to hunt around for a resource that may or may not exist.

This isn't perfect but it's a start. Gonna update it asap.

1. Choose a starting URL for a crawler (Google, Wikipedia, YouTube, or any website).
2. Request the webpage HTML source; extract English text and undetermined nouns from it and store them.
3. Continue to crawl URLs present in the webpage as long or as little as you want.

an_a.py

import scrapy
import spacy
from scrapy.exceptions import CloseSpider
class AnACrawler(scrapy.Spider):
    name = "ana_crawler"
    start_urls = [
        "https://www.wikipedia.org",
    ]
    visited_urls = set()  # set to store visited URLs
def parse(self, response):
    # Load the NLP model
    nlp = spacy.load("en_core_web_sm")

    # Extract the text and detect the language of each sentence
    doc = nlp(response.text)

    # Keep only the English sentences and extract the nouns with indefinite articles
    nouns = []
    for sent in doc.sents:
        if sent._.language["language"] == "en":
            for token in sent:
                if token.pos_ == "NOUN" and token.text in ["a", "an"]:
                    nouns.append((token.text, token.nbor().text))

    # Save the text and nouns to a file
    with open("text_and_nouns.txt", "a") as f:
        f.write(response.text + "\n")
        f.write(str(nouns) + "\n")

    # Follow hyperlinks to crawl more pages
    for a in response.css("a"):
        url = a.attrib["href"]
        if url not in self.visited_urls:
            self.visited_urls.add(url)
            yield scrapy.Request(url, self.parse)