Translation memory software for Cornish

One of the discussions I was having with Mark Trevethan by email recently was about the translation service of the Cornish Language Office, and the idea of 'translation memory', that is when text is to be translated, to store examples of previous work done. This has two main advantages, one being saving labour, and secondly improving consistency.

I had an idea to make a rudimentary version of this myself, using the Python Natural Language Toolkit. To make this work, I needed a bilingual corpus, which had the same sentences in both Cornish and English.

The electronic version of the Cornish language textbook Skeul an Yeth 1 by Wella Brown, has been made available online free by Kesva an Taves Kernewek (The Cornish Language Board).

This contains a list of example sentences at the end of every chapter, which provides the bilingual corpus for this work.

What the program does is to ask for an input sentence (currently only via the command-line) in English, and then find the 'bigrams' and 'trigrams' in it, and also do so for the sentences from Skeul an Yeth 1.

The program uses the NLTK 'stopwords' corpus, to filter the bigrams/trigrams for whether they are in a list of common words that may not have much in the way of lexical content. Sentences containing trigrams containing at least 1 non-stopword are listed first, followed by bigrams with at least 1 non-stopword, followed by trigrams and bigrams that consist solely of stopwords.

For a larger corpus the numbers of sentences found for common bigrams such as ('in', 'the') could become very large.


python kovtreylyans.py
Enter an English sentence
The cat is sleeping on the floor next to the fire.

trigrams for input sentence are:
[('the', 'cat', 'is'), ('cat', 'is', 'sleeping'), ('is', 'sleeping', 'on'), ('sleeping', 'on', 'the'), ('on', 'the', 'floor'), ('the', 'floor', 'next'), ('floor', 'next', 'to'), ('next', 'to', 'the'), ('to', 'the', 'fire'), ('the', 'fire', '.')]

bigrams for input sentence are:
[('the', 'cat'), ('cat', 'is'), ('is', 'sleeping'), ('sleeping', 'on'), ('on', 'the'), ('the', 'floor'), ('floor', 'next'), ('next', 'to'), ('to', 'the'), ('the', 'fire'), ('fire', '.')]

Listing N grams with a minimum of 1 non-stopword each:
Common trigrams:

Yma an gath a'y growedh war an leur yn-dann an gador y'n esedhva. -- The cat is lying on the floor under the chair in the sitting room. (the cat is), (on the floor)
Ottena! An maw moen na ryb an daras. -- There look! That thin boy next to the door. (next to the)
War an leur yn-dann dha weli yn dha jambour, dell vydh usys! -- On the floor under your bed in your bedroom, as usual! (on the floor)
Usi! Hag yma an gath ena ynwedh. -- Yes! And the cat is there also. (the cat is)
Nag esons! Yma an ki war an leur mes yma an gath y'n wydhenn. -- No! The dog is on the ground but the cat is in the tree. (the cat is)
Gorr glow war an tan. Oer yw hi. -- Put coal on the fire. It's cold. (the fire.)
Esedh orth an tan! Ty a vydh toemma ena. -- Sit at the fire. You will be warmer there. (the fire.)
Dewgh orth an tan! Oer yw hi! -- Come to the fire it's very cold! (to the fire)

Common bigrams:

An gath a gosk war an gweliow. -- The cat sleeps on the beds. (the cat), (on the)
Yma Jerri ow koska lemmyn. -- Jerry is sleeping now. (is sleeping)
Ple'ma an gath? -- Where is the cat? (the cat)
Usi an gath y'n lowarth? -- Is the cat in the garden? (the cat)
orth an tan -- at the fire (the fire)
A esedhons i orth an tan pub gorthugher? -- Do they sit at the fire every evening? (the fire)

Other N grams containing only stopwords:
Common trigrams:


Common bigrams:

Ni a dhybris li. Ena ni a gerdhas. Kerdh hir o dhe'n kerrek war an hal -- We ate lunch. Then we walked. It was a long walk to the rocks on the moor. (to the), (on the)
Eus jynn-skrifa war an desk? -- Is there a typewriter on the desk? (on the)
Ottena - yma an genter war an eurlenn. -- Look there - there's the nail on the carpet. (on the)
Yma pras war an woen hag yma chi ryb an pras na. -- There's a field on the down and there's a house by that field. (on the)
Sur, yma lyver war an voes. -- Certainly there is a book on the table. (on the)
Eus traow gesys war an lestrier? -- Are there things left on the dresser? (on the)
Yma bleujyow byw gesys war an fordh omma. -- There are live flowers left on the road here. (on the)
Yma padell blos war voes an gegin. -- There's a dirty pan on the kitchen table. (on the)
Ottena teyr delenn rudh war an leur. -- Look there are three red leaves on the ground. (on the)
Eus hwetek plat byghan war an lestrier? -- Are there sixteen small plates on the dresser? (on the)
Yw. Yma hi war an voes y'n gegin. -- Yes. It's on the kitchen table. (on the)
Eus amanenn war an bara? Eus! -- Is there butter on the bread? Yes! (on the)
Deves yw tanow war an voen. -- Sheep are scarce on the down. (on the)
war an amari -- on the cupboard (on the)
A nyns usi an boes war an voes hwath? -- Isn't the food on the table yet? (on the)
Esons i war an voes? -- Are they on the table? (on the)
War an voes (yma) martesen. -- On the table (it is) perhaps. (on the)
Nebes fordhow y'n ynys yw ledan lowr mes meur a fordhow ena yw re gul. -- Few roads on the island are wide enough but many roads there are too narrow. (on the)
War an voes ymons. -- They are on the table. (on the)
Skrifewgh hanow an lyver war gynsa linen an folenn! -- Write the name of the book on the first line of the page! (on the)
Esesta war an treth? -- Were you on the beach? (on the)
Esewgh hwi war an treth? -- Were you on the beach? (on the)
Y'n koes yth esa del gell war an leur. -- In the wood there were brown leaves on the ground. (on the)
An vamm re worras an kinyow war an voes lemmyn. Kynsa yma kowl onyon. -- Mother has put the dinner on the table now. First there is onion soup. (on the)
War an voes y hworrons i an boes. -- On the table they put the food. (on the)
Ena y hworrav ow hota war an gador. -- Then I put my coat on the chair. (on the)
Yma krys ow kregi war benn an gweli. -- There is a shirt hanging on the end of the bed. (on the)
Gorr an kellylli war an voes! -- Put the knives on the table! (on the)
Y'n seythves dydh an dra o dien. -- On the seventh day the matter was complete. (on the)
Ny yllydh jy esedha war an glesin. Re lyb yw ev. -- You can't sit on the lawn. It's too wet. (on the)
Ev a redyas y hanow y'n peswara koloven war an pympes folenn a'n paper-nowodhow. -- He read his name in the fourth column on the fifth page of the newspaper. (on the)
Pan splann an loergann war an arvor a-dreus an mor kosel, assyw hi teg! -- When the full moon shines on the shore across the calm sea, how beautiful it is! (on the)
Tasik! Tasik! Ottena! Ergh war an glesin! -- Daddy! Daddy! Look! Snow on the lawn! (on the)
War drysa estyllenn an argh-lyvrow y'n esedhva yma, dell dybav. -- On the third shelf of the bookcase in the lounge it is, I think. (on the)
Nyns eus karr vyth y'n fordh. -- There is no car at all on the road. (on the)
An gewer yw hager war an heyl. Ny yll den gweles a-dreus dhodho. -- The weather is ugly on the estuary. A person cannot see across it. (on the)
An rewler a worras an lytherow war an desk rybdho. -- The manager put the letters on the desk beside him. (on the)
Ottena! A-dro dhe hanterkans hos war an lynn yn kres an hal. -- Look there! About fifty ducks on the lake in the middle of the moor. (on the)
An peswara drehevyans diworto yw ev a'n keth tu. -- It's the fourth building from it on the same side. (on the)
Goel Sen Pyran a vydh pub blydhen dhe'n pympes a vis Meurth. -- St Piran's Day is on the fifth of March each year. (on the)
Henri a vynn esedha war an isella kador. -- Henry will sit on the lowest chair. (on the)
Ny yll ev esedha war an ughella huni. -- He cannot sit on the highest one. (on the)
Prag y tregh ev an skorrennow na? Drefenn ev dh'aga leski war an tansys. -- Why does he cut those branches? Because he burns them on the bonfire. (on the)
'Yma diwros war an fordh ena ha gour shyndys a'y wrowedh war an leur', an gwithyas kres a leveris. 'Res yw dhis gortos deg mynysenn, mar pleg. Ni a vynn y worra dhe'n klavji a-dhistowgh.' -- 'There's a bicycle on the road there and a man lying injured.' replied the policeman. 'You must wait ten minutes, please. We will take him to hospital immediately.' (on the)
Kerdh hir yw dhe'n eglos. -- It's a long walk to the church. (to the)
Py chambour yw an nessa dhe'n lowarth a-rag? -- Which bedroom is nearest to the front garden? (to the)
Ke dhe'n fenester, mar pleg! -- Go to the window, please! (to the)
Nyns yw an traow ma pur haval orth an re erell, yns i? -- These are not very similar to the others, are they? (to the)
Martyn eth dhe'n treth mes nyns eth dhe neuvya. -- Martin went to the beach but he didn't go to swim. (to the)
Y'n eur na yth eth ev dhe skol an eglos. -- He then went to the church school. (to the)
My a wra lenna hwedhel dhe'n fleghes pub gorthugher. -- I read to the children every evening. (to the)
An dowr a yn nans dhe'n mor. -- The water goes down to the sea. (to the)
Ni oll warbarth eth yn-nans dhe'n treth rag neuvya. -- We all went down to the beach together in order to swim. (to the)
An keur a ganas dhe'n fleghes. -- The choir sang to the children. (to the)
A vynnowgh hwi mones genen dhe'n dons? -- Will you go with us to the dance? (to the)
Dowr an fenten a dhe'n gover. -- The spring water goes to the brook. (to the)
Karol a lanhas an lestri kyns aga daskorr dhe'n lestrier. -- Carol cleaned the dishes before returning them to the dresser. (to the)
An tiek a dhros y vughes dhe'n skiber. -- The farmer brought his cows to the barn. (to the)
An brassa stevell yw an nessa stevell dhe'n wolghva. -- The biggest room is the nearest room to the bathroom. (to the)
An skoloryon, mebyon ha mowesi, a dhe'n keth skol y'n dre. -- The schoolchildren, boys and girls, go to the same school in town. (to the)
An awel o krev. Ny allas an gorholyon dos ogas dhe'n porth. -- The wind was strong. The ships couldn't come near to the harbour. (to the)