B. To Do

Words in bold indicate how you should refer to the item when discussing it with me or others.

B.1 Things that need to be done

These items need to be done before I consider Aspell finished. Unfortunately, I do not have the time to do all so if you are interested in helping me with one of these tasks please email me. Good C++ skills are needed for most of these tasks involving coding.

• Convert manual from LyX/L^ATEX to Texinfo.
• Clean up copyright notices and bring the Aspell package up to GNU Standards.
• Add gettext support to Aspell.
• Allow Aspell to check documents which are in UTF-8. The main thing to be done is to get Aspell to display UTF-8 correctly. This will involve being able to get the length of UTF-8 strings and display UTF-8 using the curses library.
• Make Aspell Thread safe. Even though Aspell itself is not multi-threaded I would like it to be thread safe so that it can be used by multi-threaded programs. There are several areas of Aspell that that are potently thread unsafe (such as accessing a global pool) and several several classes which have the potential of being used by more than one thread (such as the personal dictionary).
• Allow filters to be loaded at run-time. At the moment all filters must be complied in.
• Integrate Kevin Hendricks affix compression code into Aspell. His code is already in use in OpenOffice as part of the lingucomponent component. More information can be found at http://whiteboard.openoffice.org/lingucomponent/dictionary.html. The latest version of his code is also available there.
• Create a C++ interface for Aspell, possibly on top of the C one.
• Enhance ispell.el so that it will work better with the new Aspell.

B.2 Good ideas that are worth implementing

These items all sound like good ideas however I am not sure when I will get to implementing then if ever.

• Use Lawrence Philips' new Double Metaphone algorithm. See http://aspell.sourceforge.net/metaphone/.
• Come up with a plug-in for gEdit the gnome text editor.
• Change languages (and thus dictionaries) based on the information in the actual document.
• Come up with a nroff mode for spell checking.
• Come up with a mode that will skip words based on the symbols that (almost) always surround the word. (Word skipping by context)
• Create two server modes for Aspell. One that uses the DICT protocol and one that uses ispell -a method of communication of some arbitrary port.
• Come up with thread safe personal dictionaries.
• Use the Hidden Markov Model to base the suggestions on not only the word itself but on the context around the word.
• Having a way to email the personal dictionary and/or replacement list to a particular address either periodical or when it grows to a certain size.
• Be able to accept words with spaces in them as many languages have words such as as a word in a foreign phrases which only make sense when followed by other words.

The following good ideas where found in the ispell WISHES file so I thought I would pass them on.

• Ispell should be smart enough to ignore hyphenation signs, such as the TEX \- hyphenation indicator.
• (Jeff Edmonds) The personal dictionary should be able to remove certain words from the master dictionary, so that obscure words like "wether" wouldn't mask favorite typos.
• (Jeff Edmonds) It would be wonderful if ispell could correct inserted spaces such as "th e" for "the" or even "can not" for "cannot".
• Since ispell has dictionaries available to it, it is conceivable that it could automatically determine the language of a particular file by choosing the dictionary that produced the fewest spelling errors on the first few lines.

B.3 Notes and Status of various items

B.3.1 General region skipping

I want to implement this give other people an idea of how it should be done and because I am really sick of having to spell check through url and email address.

B.3.2 Word skipping by context

This was posted on the Aspell mailing list on January 1, 1999:

I had an idea on a great general way to determine if a word should be skipped. Determine the words to skip based on the symbols that (almost) always surround the word.

For example when asked to check the following C++ code:

cout « "My age is: " « num « endl;  
cout « "Next year I will be " « num + 1 « endl; 

cout, num, and endl will all be skipped. "cout" will be skipped because it is always preceded by a «. "num" will be skipped because it is always preceded by a «. And "endl" will be skipped because it is always between a « and a ;.

Given the following html code.

<table width=50% cellspacing=0 cellpadding=1>  
<tr><td>One<td>Two<td>Three  
<tr><td>1<td>2<td>3  
</table> 
 
<table cellspacing=0 cellpadding=1>  
</table>

table, width cellspacing, cellpadding, tr, td will all be skipped because they are always enclosed in "<>". Now of course table and width would be marked as correct anyway however there is no harm in skipping them.

So I was wondering if anyone on this list has any experience in writing this sort of context recognition code or could give me some pointers in the right direction.

This sort of word skipping will be very powerful if done right. I imagine that it could replace specific spell checker modes for Tex, Nroff, SGML etc because it will automatically be able to figure out where it should skip words. It could also probably do a very good job on programming languages code.

If you are interested in helping be out with this or just have general comments about the idea please let me know.

B.3.3 Hidden Markov Model

Knud Haugaard Sørensen suggested this one. From his email on the Aspell mailing list:

consider this examples.

a fone number. -> a phone number.
a fone dress. -> a fine dress.

the example illustrates that the right correction might depend on the context of the word. So I suggest that you take a look on HMM to solve this problem.

This might also provide a good base to include grammar correction in aspell.

see this link http://www.cse.ogi.edu/CSLU/HLTsurvey/ch1node7.html

I think it is a great idea. However unfortunately it will probably be very complicated to implement. Perhaps in the far future.

B.3.4 Email the Personal Dictionary

Some one suggest in a personal email:

Have you thought of adding a function to aspell, that - when the personal dictionary has grown significantly - sends the user's personal dictionary to the maintainer of the corresponding aspell dictionary? (if the user allows it)

It would be a very useful service to the dictionary maintainers, and I think most users can see their benefit in it too.

And I replied:

Yes I have considered something like that but not for the personal dictionaries but rather the replacement word list in order to get better test data for http://aspell.sourceforge.net/test/. The problem is I don't know of a good way to do this sense Aspell can also be used as a library. It also is not a real high priority, especially sense I would first need to learn how to send email within a C++ program.

B.3.5 Words With Spaces in Them

While this is something I would like to do it is not a simple task. The basic problem is that when tokenizing a string there is no good way to keep phrases together. So the solution is to some how add special conditions to certain words which will dictate which words can come before/after it. Then there is also a problem of how to come up with intelligent suggestions. What further complicates things is that many applications send words to Aspell a word at a time. So even if Aspell did support such a thing many applications that would use Aspell will not. So, in order for this to work applications will need to send text to Aspell a document or at least a sentence at a time. Unfortunately the framework for doing this is not there yet. It will be once I finish the filter interface. Another possible is to provide call back functions in which Aspell will be able to request the previous or next word on request. Yet again the framework for doing this is not there. Perhaps sometime in the near future.