sunnuntai 11. toukokuuta 2014
Bad and worse code in scientific programming
I was just reading a couple of articles about software engineering: The Low Quality of Scientific Code and Why Bad Scientific Code Beats Code Following Best Practices. Some of you may remember me from such presentations as FSCONS 2013, where I talked about the very same thing. I very much agree with the first text of course, most code I have to look at is rather dreadful, and it is very much a surprise it ever works. And one of the things I've learnt after moving to new projects from HFST (and apertium) is that, while there's a lot of bad code in there, it's actually still among the better ones with all the floss software engineering conventions that actually got implemented and at least someone every once in a while cares to follow. Take for example my trying to learn statistical machine translation, the most prominent project in the field is called moses. No tests, cannot be installed, consists mostly of kludgy scripts that only work occasionally, often by a side effect of some other script ran before in the same directory. When you want to use moses from other project you make a note of where you unpacked its source and hope that everyone who uses it will have same random scripts in same places, in one of the billion script directories there is. The thing is, it's not much harder to actually do things properly, you don't need to hire a software engineer to understand that there should be a test that runs your program and give it some input and gets the expected output and doesn't crash and all simple things like that. It's not intellectually difficult thing to grasp and the implementation doesn't take more than few minutes which is saved in each of the update and debugging cycles. So yeah, I don't have much to rant now, the blogs already said the things.
lauantai 3. toukokuuta 2014
Some EACL ideas
So I went to EACL in Gothenburg without having anything much to present there. One of the things that I usually do when I have to fly around the world with long layovers is update my compling projects. Maybe it’s inspired by Norvig’s spell-checker, who knows. This time I of course spent time on making apertium-fin-swe. The array of apertium-based Finnish translators is starting to shape up to be nice it is. This invokes the traditional problem that there’s no reuse of code in NLP, and there are no standards for whatever analyses are.
keskiviikko 16. huhtikuuta 2014
Unsupervised cycling in Nepal
Last week I attended to a conference in Nepal called CICLing. This is my second time in the conference, last being 2012 in Delhi, so I knew something what to expect, both in terms of the conference and the country, Nepal after all is not so different from India (though it is! But more of that later). The conference itself is reasonably sized, such that you can meet most of the active participants, there's only one track and most people will usually follow or stay in vicinity. Lots of social events, focus is at least somewhat on linguistics and applications too, not just engineering. Might sound a bit suspicious to waste money to go to faraway places to socialise, but the amount of support for local native languages and contacts is good, so it's well within reason.
Ok, the first impression of the conference is definitely unsupervised. I think at least 33 % of the presentations and posters will have contained that word. As it is, there's nothing wrong in unsupervised methods, if they do what the promise is it's all fine, if there's no supervised way to do the same, good. However, it's not always the case. One of the main promise of unsupervised methods is that they are cheap and easy and require no experts. But if you've spent 3 years coming up with unsupervised methods, is that not work, and very expensive expert work at it? But maybe it's language independent and we get rest of the world's languages for free? I've rarely seen that to be the case though, there's always something that a native speaking informant needs to explain, why are these words being split here, why's that disappearing. For tasks that I'm familiar, this time with informant trying to tell the engineer what's wrong is longer than what would it would take for one computational linguist and native informant to create same system in supervised manner with better quality. For morphology, what was presented here, it's especially obvious, the unsupervised learnings of morphology have not to date reached the quality of rule-based systems. And even so, the unsupervised systems will anyways need to be annotated by native informant and linguist, cause the morphs themselves don't tell us much, for the 99 % of the real applications, we need to know if they are words, present tense 3rd person singular affixes or question clitic particles. Building unsupervised systems for morphology in few years with recall of 80 % vs. building one in a gsoc period of 3 months with 95 % recall, come on now really, which one seems more sane? But unsupervised learning it's an interesting question, to paraphrase one answer, and it is indeed, but interesting like solving a hard sudoku puzzle without lookahead, or interesting like trying to throw darts blindfolded with hands tied and still reaching 501 sharp, in finite time, not interesting as in useful to scientific community. That being said, I fully support the goal when it's formulated as a consistency check for the rule-based system, that will be useful indeed, since humans are prone to errors, likely to make wrong generalisations, miss some generalisations, all that, a statistically oriented system for morphology might aiui reveal this kind of errors.
Not to totally just bash unsupervised crowd, another keynote speaker did present slides about common sense knowledge system. They reminded me of what I'd learnt while studying language technology in University of Joensuu–some 10 years ago–on a course called History of Language Technology (and if it weren't in a book called failed projects, it should've been). This project has all the interesting things there are in language technology. Computers understanding common sense. Logical formulas. Inferences, deductions, networks, seems very charming and the way it should be. Then you realise that each of the formulas and networks cover just one very small part of the world, and these are all hand-build formulas, so there must be millions of them, and they are nothing else than English words written in capitals, with subjects and objects of sentences rearranged to look like mathematical functions, with some quantifiers sprinkled around. And you can collect those for a life time to cover 1 % of the common sense. And that's quite possibly the main reason engineers want to get rid of linguists, so that nobody would ever start building those systems. Though I don't know if we can unsupervised learn those either.
With smaller presentations, there was well enough good things for me to find that trip was worth it. Few uses of morphological segmentation in machine translation, Kazakh spell-checking and Bishnupriya Manapuri FST system for example. If I ever have time to follow them up. And some more. Chunking and bilingual dictionaries building.
Outside scientific festivals, Nepal was quite positive surprise, I expected something like India, but it's actually a nicer version of India, more tourist-friendly and less crowded. A bunch of nice bars and shops in backpacker ghetto, hawkers and scammers won't bother you after one or two sharp no's. And there was an intercollege music contest which seemed to feature mainly metal, with three headliners. Audience was cool, nice pitting and all. Music wasn't all that bad. We never solved the structure of Vomiting Snakes. And unusually nice group of conference people for such social events after official events that I met.
My slides are at the usual place in my github repo, with a questions doge. That wasn't all that succesfull, but I've already used up the whole spell-checking crud, so let's get to see what world has to offer.
Ok, the first impression of the conference is definitely unsupervised. I think at least 33 % of the presentations and posters will have contained that word. As it is, there's nothing wrong in unsupervised methods, if they do what the promise is it's all fine, if there's no supervised way to do the same, good. However, it's not always the case. One of the main promise of unsupervised methods is that they are cheap and easy and require no experts. But if you've spent 3 years coming up with unsupervised methods, is that not work, and very expensive expert work at it? But maybe it's language independent and we get rest of the world's languages for free? I've rarely seen that to be the case though, there's always something that a native speaking informant needs to explain, why are these words being split here, why's that disappearing. For tasks that I'm familiar, this time with informant trying to tell the engineer what's wrong is longer than what would it would take for one computational linguist and native informant to create same system in supervised manner with better quality. For morphology, what was presented here, it's especially obvious, the unsupervised learnings of morphology have not to date reached the quality of rule-based systems. And even so, the unsupervised systems will anyways need to be annotated by native informant and linguist, cause the morphs themselves don't tell us much, for the 99 % of the real applications, we need to know if they are words, present tense 3rd person singular affixes or question clitic particles. Building unsupervised systems for morphology in few years with recall of 80 % vs. building one in a gsoc period of 3 months with 95 % recall, come on now really, which one seems more sane? But unsupervised learning it's an interesting question, to paraphrase one answer, and it is indeed, but interesting like solving a hard sudoku puzzle without lookahead, or interesting like trying to throw darts blindfolded with hands tied and still reaching 501 sharp, in finite time, not interesting as in useful to scientific community. That being said, I fully support the goal when it's formulated as a consistency check for the rule-based system, that will be useful indeed, since humans are prone to errors, likely to make wrong generalisations, miss some generalisations, all that, a statistically oriented system for morphology might aiui reveal this kind of errors.
Not to totally just bash unsupervised crowd, another keynote speaker did present slides about common sense knowledge system. They reminded me of what I'd learnt while studying language technology in University of Joensuu–some 10 years ago–on a course called History of Language Technology (and if it weren't in a book called failed projects, it should've been). This project has all the interesting things there are in language technology. Computers understanding common sense. Logical formulas. Inferences, deductions, networks, seems very charming and the way it should be. Then you realise that each of the formulas and networks cover just one very small part of the world, and these are all hand-build formulas, so there must be millions of them, and they are nothing else than English words written in capitals, with subjects and objects of sentences rearranged to look like mathematical functions, with some quantifiers sprinkled around. And you can collect those for a life time to cover 1 % of the common sense. And that's quite possibly the main reason engineers want to get rid of linguists, so that nobody would ever start building those systems. Though I don't know if we can unsupervised learn those either.
With smaller presentations, there was well enough good things for me to find that trip was worth it. Few uses of morphological segmentation in machine translation, Kazakh spell-checking and Bishnupriya Manapuri FST system for example. If I ever have time to follow them up. And some more. Chunking and bilingual dictionaries building.
Outside scientific festivals, Nepal was quite positive surprise, I expected something like India, but it's actually a nicer version of India, more tourist-friendly and less crowded. A bunch of nice bars and shops in backpacker ghetto, hawkers and scammers won't bother you after one or two sharp no's. And there was an intercollege music contest which seemed to feature mainly metal, with three headliners. Audience was cool, nice pitting and all. Music wasn't all that bad. We never solved the structure of Vomiting Snakes. And unusually nice group of conference people for such social events after official events that I met.
My slides are at the usual place in my github repo, with a questions doge. That wasn't all that succesfull, but I've already used up the whole spell-checking crud, so let's get to see what world has to offer.
maanantai 24. maaliskuuta 2014
The first weeks in Ireland
This blog is a continuation to my hugely successful squiggly colorful underlinings blog I had in University of Helsinki. Since I've now got my PhD and finished drawing red underlines, not to mention that the University of Helsinki will more or less likely remove my access now that I don't live there anymore. Ok, so this first blog post is not going to contain science, much linguistics or anything, so you are expecting that you can skip to the rest of this. Just a rant of the good and the bad in moving from (a University job in) Finland to (a University job in) Ireland. I'll resume the normal order once I get back to doing science, not to worry.
Tilaa:
Blogitekstit (Atom)