Skrifennow

My blog, imported from Blogger and converted using Jekyll.

Syllable segmentation in Cornish - forward vs. backward segmentation

Aug 14, 2016

The syllable segmentation module of TaklowKernewek I have commented on earlier in this blog, and on my website.

However there is much more to discuss, and one aspect of this is that the program offers a choice between forwards and backwards segmentation.

This means either starting from the beginning of the word, and working forwards assigning the letters to particular syllables, or starting from the end and working backwards.

I present some of the code from the program, which is admittedly difficult to read, and if you like, skip down to the examples at the bottom. It may also be easier to read at my Bitbucket site.

The core of this program is a set of regular expressions, as follows:

# syllabelRegExp should match syllable anywhere in a word
# a syllable could have structure CV, CVC, VC, V
# will now match traditional graphs c-, qw- yn syllable initial position
syllabelRegExp = r'''(?x)
((bl|br|Bl|Br|kl|Kl|kr|Kr|kn|Kn|kwr?|Kwr?|qwr?|Qwr?|ch|Ch|Dhr?\'?|dhr?\'?|dl|dr|Dr|fl|Fl|fr|Fr|vl|Vl|vr|Vr|vv|ll|gwr?|gwl?|gl|gr|gg?h|gn|Gwr?|Gwl?|Gl|Gr|Gn|hwr?|Hwr?|ph|Ph|pr|pl|Pr|Pl|shr?|Shr?|str?|Str?|skr?|Skr?|skw?|Skw?|sbr|Sbr|spr|Spr|sp?l?|Sp?l?|sm|Sm|tth|Tth|thr?|Thr?|tr|Tr|tl|Tl|wr|Wr|wl|Wl|[bckdfjvlghmnprstwyzBCKDFJVLGHMNPRSTVWZY]) # consonant
\'?(ay|a\'?w|eu|ey|ew|iw|oe|oy|ow|ou|uw|yw|[aeoiuy])\'? #vowel
(lgh|ls|lt|bl|br|bb|kl|kr|kn|kwr?|kk|n?ch|dhr?|dl|n?dr|dd|fl|fr|ff|vl|vv|gg?ht?|gw|gl|gn|ld|lf|lk|ll|mm|mp|nk|nd|nj|ns|nth?|nn|ph|pr|pl|pp|rgh?|rdh?|rth?|rk|rl|rv|rm|rn|rr|rj|rf|rs|sh|st|sk|ss|sp?l?|tt?h|tt|[bdfgljmnpkrstvw])? # optional const.
)| # or
(\'?(ay|a\'?w|eu|ew|ey|iw|oe|oy|ow|ou|uw|yw|Ay|Aw|Ey|Eu|Ew|Iw|Oe|Oy|Ow|Ou|Uw|Yw|[aeoiuyAEIOUY])\'? # vowel
(lgh|ls|lt|bl|bb|kl|kr|kn|kwr?|kk|cch|n?ch|dhr?|dl|n?dr|dd|fl|fr|ff|vl|vv|gg?ht?|gw|gl|gn|ld|lf|lk|ll|mm|mp|nk|nd|nj|ns|nth?|nn|ph|pr|pl|pp|rgh?|rdh?|rth?|rk|rl|rv|rm|rn|rr|rj|rf|rs|sh|st|sk|ss|sp?l?|tt?h|tt|[bdfgljmnpkrstvw]\'?)?) # consonant (optional)
'''
# diwethRegExp matches a syllable at the end of the word
diwetRegExp = r'''(?x)
((bl|br|Bl|Br|kl|Kl|kr|Kr|kn|Kn|kwr?|Kwr?|qwr?|Qwr?|ch|Ch|Dhr?\'?|dhr?\'?|dl|dr|Dl|Dr|fl|Fl|fr|Fr|vl|Vl|vr|Vr|vv|ll|gwr?|gwl?|gl|gr|gg?h|gn|Gwr?|Gwl?|Gl|Gr|Gn|hwr?|Hwr?|ph|Ph|pr|pl|Pr|Pl|shr?|Shr?|str?|Str?|skr?|Skr?|skw?|Skw?|sbr|Sbr|spr|Spr|sp?l?|Sp?l?|sm|Sm|tth|Tth|thr?|Thr?|tr|Tr|tl|Tl|wr|Wr|wl|Wl|[bckdfjlghpmnrstvwyzBCKDFJLGHPMNRSTVWYZ]\'?)? #consonant or c. cluster
\'?(ay|a\'?w|eu|ew|ey|iw|oe|oy|ow|ou|uw|yw|Ay|Aw|Ey|Eu|Ew|Iw|Oe|Oy|Ow|Ou|Uw|Yw|\'?[aeoiuyAEIOUY]\'?) # vowel
(lgh|ls|lt|bl|br|bb|kl|kr|kn|kwr?|kk|cch|n?ch|dhr?|dl|n?dr|dd|fl|fr|ff|vl|vv|gg?ht?|gw|gl|gn|ld|lf|lk|ll|mm|mp|nk|nd|nj|ns|nth?|nn|ph|pr|pl|pp|rgh?|rdh?|rth?|rk|rl|rv|rm|rn|rr|rj|rf|rs|sh|st|sk|ss|sp?l?|tt?h|tt|[bdfgjklmnprstvw]\'?)? # optionally a second consonant or cluster ie CVC?
(\-|\.|\,|;|:|!|\?|\(|\))*
)$
'''
# kynsaRegExp matches syllable at beginning of a word
# 1st syllable could be CV, CVC, VC, V
kynsaRegExp = r'''(?x)
^((\'?(bl|br|Bl|Br|kl|Kl|kr|Kr|kn|Kn|kwr?|Kwr?|qwr?|Qwr?|ch|Ch|Dhr?|dhr?|dl|dr|Dr|fl|Fl|fr|Fr|vl|Vl|vr|Vr|gwr?|gwl?|gl|gr|gn|Gwr?|Gwl?|Gl|Gr|Gn|hwr?|Hwr?|ph|Ph|pr|pl|Pr|Pl|shr?|Shr?|str?|Str?|skr?|Skr?|skw?|Skw?|sbr|Sbr|spr|Spr|sp?l?|Sp?l?|sm|Sm|tth|Tth|thr?|Thr?|tr|Tr|tl|Tl|wr|Wr|wl|Wl|[bckdfghjlmnprtvwyzBCKDFGHJLMNPRTVWYZ])\'?)? # optional C.
\'?(ay|a\'?w|eu|ew|ey|iw|oe|oy|ow|ou|uw|yw|Ay|Aw|Ey|Eu|Ew|Iw|Oe|Oy|Ow|Ou|Uw|Yw|[aeoiuyAEIOUY])\'? # Vowel
(lgh|ls|lk|ld|lf|lt|bb?|kk?|cch|n?ch|n?dr|dh|dd?|ff?|vv?|ght|gg?h?|ll?|mp|mm?|nk|nd|nj|ns|nth?|nn?|pp?|rgh?|rdh?|rth?|rk|rl|rv|rm|rn|rj|rf|rs|rr?|sh|st|sk|sp|ss?|tt?h|tt?|[jw]\'?)? # optional C.
(\-|\.|\,|;|:|!|\?|\(|\))*
)'''


In the actual segmentation of the word itself, the expressions kynsaRegExp and diwetRegExp are used, depending on whether we are going forwards starting from the beginning or backwards from the end:


if fwds:
# go forwards
sls = rannans.ranna_syl(self.graph,regexps.kynsaRegExp,fwd=True,bwd=False)
else:
# go backwards from end
sls = rannans.ranna_syl(self.graph,regexps.diwetRegExp,fwd=False,bwd=True)


where ranna_syl() is the actual function that returns a list of syllables from the word ger:


def ranna_syl(self,ger,regexp,fwd=True,bwd=False):
""" divide a word into a list of its syllables
and return this as a list of plain text strings
"""
syl_list = []
if fwd:
# go forwards through the word
while ger:
# print(ger)
k = self.match_syl(ger,regexp)
# print("kynsa syl:{k}".format(k=k))
# add the syllable to the list
if k != '':
syl_list.append(k)
if k != '' and len(ger.split(k,1))>1:
# if there is more of the word after the
# 1st syllable
# remove the 1st syllable
ger = ger.split(k,1)[1]

else:
ger = ''
if bwd:
# go backwards from the end through the word
while ger:
# print(ger)
d = self.match_syl(ger,regexp)
# print(d)
# add the syllable to the list
if d != '':
syl_list.insert(0,d)
if d != '' and len(ger.rsplit(d,1))>1:
# if there is more of the word before the
# last syllable
# remove the last syllable
ger = ger.rsplit(d,1)[0]
else:
ger = ''
# this is returning
# a list of plain text
# not Syllabenn objects
return syl_list


The syllabelRegExp regular expression is used in Syllabenn class itself, as part of the code that initates a Syllabenn object and works out the syllable parts, i.e. consanant clusters and vowels, and the overall length.

Example sentences

The effect of going forwards or backwards can be illustrated in the processing of an example sentence:

Going backwards from the end, tends to maximise consonants at the beginning of syllables. For example the word 'gewer' is processed into ['ge', 'wer'] i.e. the w is assigned to the second syllable whereas in this word the 'ew' is actually pronounced as a diphthong. The gemminated consonant 'mm' in lemmyn is split into two different syllables.
Now working forward, the processing of the word 'gewer' now splits into ['gew', 'er'] which accords with the status of 'ew' as a diphthong. 'Lemmyn' now splits into ['lemm', 'yn'] assigning the whole of the gemminated consonant to the first syllable. The word 'Fatell' now has the 't' assigned to the first syllable

A similar effect can be seen in another sentence:
Special cases such as the unstressed monosyllables 'ha', and 'dell' are detailed in the file datageryow.py.

With forwards segmentation, the processing of 'kommolek', and 'hevel' assigns consonants to the coda of syllables rather than maximising the onset.

An update on TaklowKernewek - Cornish language Python tools

Aug 13, 2016

I have been continuing a little more development on my Cornish language processing tools.

I have added recently a number of things to them, including reverse mutation, and a launcher program. In this post, I will describe a little of 'Niverow', a program to write out numbers in Cornish, 'Mutatya' - a program to generate mutated forms of words, and 'Inflektya', a program to generate inflected verb forms.

Firstly the launcher program. This is the file  TaklowKernewekLonchyer.pyw
which itself imports a couple of other scripts TaklowKernewekLonch.py and launchmodes.py which are largely based on example scripts from Programming Python 3rd edition by Mark Lutz.

Running this brings up a basic array of buttons:
Clicking on any of these buttons will bring up its own GUI window.

 Niverow

In the boxes, specify the number, and the noun, and noun plural, if you wish to include the noun. It is also necessary to tick the box next to "Usya Hanow" to do so. Tick "Hanow Benow" to indicate that it is a feminine noun, which will make the program use the feminine forms of numbers (e.g. diw in place of dew) where they are needed. Decimal numbers can be used, and this program treats them reading out one digit at a time.
If using a decimal number, the program will use the number + a + plural noun format. The plural is assumed to be the noun + -ow unless a different one is specified.
This format is also used if the number has more than three elements

Mutatya

I have mentioned mutatya.py before on this blog,  but it has a few new features including an option to use the traditional spelling forms as used in the SWF Traditional variant. These consist of using c instead of k before some vowels, and using qw- instead of kw- and wh- instead of hw-.

The other new feature is reverse mutation, taking a word and identifying what words it could originally have been. This doesn't check whether the hypothesised unmutated word actually exists, or whether the mutation is actually gramatically possible.

Traditional forms of the soft mutation c-->g, and the hard mutation gw-->qw. These would be k-->g and gw-->kw in Kernewek Kemmyn, or SWF Main Form.

The word 'garr' could either be an unmutated form, or a soft mutation of 'karr'. Likewise 'hanow' could be 'hanow', breathed mutation of 'kanow' or mixed mutation of 'ganow'.

Inflektya

This program generates inflected forms of verbs, selecting the tense in the left hand menu, the person in the next one, whether to use suffixed pronouns, and whether to expect SWF input and give SWF output (the default is Kernewek Kemmyn).
Showing the inflected forms of the regular verb 'prena' for present, preterite, and imperfect tenses.
Attempting to show the future tense of prena gives an error message, since the simple future doesn't exist for this verb.

The imperative, which doesn't exist for the impersonal, and first person forms, and is not very common except in 2nd person 'Ty' and 'Hwi' forms.
For the irregular verb 'bos' to be, the simple future is shown.
The present subjunctive of the verb 'dismygi' in Kernewek Kemmyn, and in SWF.


Welsh language internet memes

Aug 11, 2016

Here's a few captioned pictures in Welsh I put out on clecs.cymru (like Twitter, but in Welsh) previously and a couple of Cornish ones:

Based on the song Can y Cardi



Mae Powys yn wlad fawr iawn. Scattered towns separated by vast tracts of conifers...
Mae e yn Forg!
A typical summer Saturday on the main train line through Cornwall. In Cornish it is not recommended to use "war an tren", instead "y'n tren" if you are indeed travelling inside the carriage.
This may look similar to a screenshot from Poldark but it is in fact from the Cornish version of Lord of the Rings, in a scene showing the hobbits in the Old Forest (An Hen Goeswik or An Goeswik Goth).

Using QOSM to provide OpenStreetMap basemap in QGIS for my slopelines project

Aug 5, 2016

In previous posts I have described how I have used a segmented DEM with RSGISlib, a method originally developed as part of my MSc dissertation on Martian glaciers, for visualising slopes on Earth.

I have been looking at these again, and have produced a few maps with the QOSM plugin, which dynamically loads OpenStreetMap tiles into QGIS.

These maps below were produced with the Thunderforest Outdoors rendering. The slopelines are as previously, red for convex slopes, blue for concave, with thicker lines denoting steeper slopes.

Previous versions of this had used a version of OpenStreetMap that I was preparing myself from the source data within QGIS. Using the basemap tiles I can focus on adapting the visualisation of the slope lines themselves a bit more.

Bodmin Moor

Camborne and Redruth, including Carn Brea and Portreath

Penzance

Truro

A close up of Truro
For a while I thought the QGIS bug I found had recurred which meant that the arrows had disappeared. However it was something wrong in my data dependent styling, not sure how it happened. Once fixed, I show the slope direction with arrows, which is only possible up to about 1:10,000 otherwise the rendering struggles:

Indicating the downslope direction with arrows.

Delabole, showing the slate quarry.
St. Ives

Aberystwyth, Wales

Cadair Idris, Wales

A wider view of Cadair Idris, too small a scale to allow plotting the arrows.


Pen y Fan, Brecon Beacons, Wales


Retrotopia draft map

Jul 23, 2016

The author John Michael Greer has been publishing a speculative fiction series currently on his blog, The Archdruid Report called 'Retrotopia'. It is set in around 2065, in a North America that has undergone some changes after the breakup of the USA.


Here is a draft map, based on his descriptions in the comments to a recent post, showing the approximate borders of the successor states to the USA and Canada. I have yet to deal with states that are split between more that one nation in 2065.

The nations of the Retrotopia stories, not including at this time states that are split between more than one Retrotopia nation. No account of sea level rise is made, which is assumed to be 6 feet above present day levels.
I used a map of US counties from here, and US state, and Canadian province shapefiles from here. Update - I switched to using this map of Canadian provinces which includes Nunavut.

I have also used the ETOPO1 bedrock relief topographic map as a background, which shows the elevation of Greenland without the ice sheet. It is perhaps a century or two too early to assume complete deglaciation of Greenland. The portion of Greenland below sea level is not actually connected to the sea here, it would be at a +50 metre sea level as assumed in the 25th century setting of Star's Reach.

Update

I have made some updates based on further descriptions based on John Michael Greer's comments, which account for the states that are divided:

The borders of the Retrotopia nations can be seen by the colours, with present-day US states and counties shown as lines.
The Lakeland Republic is not labelled since the Free City of Chicago label has prevented it from appearing.


In this case the Lakeland Republic label wins out.

These are the borders of the western areas, as far as I understand JMGs comments.


Close-up of the north-east former USA and adjacent part of Canada.
Black and white version.


The shapefile itself containing all of the Retrotopia nations in North America I have made available here.

It is also available as a KML file suitable for use with Google Earth.

Showing 2 meters of sea level rise in blue, with 2-4 metres shaded in lighter blue.

All Posts