Writing a large project using J programming language
Disclosure
This is a "general" question, perhaps without a specific answer, but it is not intended as a flame war. I would really like some information before embarking on my project.
I have to implement a particular project which would really benefit from the data structures and abstractions provided by J. This is a large project, meant to function as the central component of a large (soft real-time) web application. So performance is very important.
I have been trying to find some information about the usage of J in large commercial or open source projects, but I am unable to find any information on which to base my decision to move forward. I have:
- Searched Google Trends, but received the following response: "Your terms - j programming language - do not have enough search volume to show graphs."
- Searched on free(code), and not found a single project using J
- Searched on Sourceforge, and not found a single project using J
- Searched on Lambda the Ultimate, and only found the following discussion that obliquely references APL
- Searched generally on Google and Bing, and failed to find any examples of large scale projects in deployment that use J
Would I be making a mistake in using J for my project? It seems to have everything--especially in terms of data structures, abstraction and concision--that I want. Sure, I could spend time simulating all those properties in F#, or C#, or C++, but J already has them, so...
Can someone please tell me some drawbacks of using J (or any obscure language) for important projects? Is it not sufficiently performant? Does it not have libraries? Anything else I should know?
Thanks in advance for your responses.
Answer
I think J is suitable for critical use on large, important projects. But I also think J would be a very adventurous choice (read: high-risk) for such a project.
The two good-sized companies I'm aware of that use J as the central technology for a major software system are Cognos and Luen Thai. (Cognos has been purchased by IBM.) Some discussion of these systems, or at least some experiences involving them, have shown up in the J Chat Forum. There's not much information there, however.
There may be other big software systems that rely on J, but I've not identified them (and I have been keeping my eye out for years.)
Dmitri Chubarov is right to think that difficulty finding appropriately skilled developers could be a significant problem. Moreover, I think it is difficult for people who have not already developed significant skill in an array language to estimate how much effort it will take them to become good at programming in J. I personally consider it particularly important that J be learned out of an attitude of joyful excitement, and this attitude can be put at risk by the production pressures of a software project. (It is possible to learn J on the job and love it, but the risk that a programmer will hate it because they "have to" learn it should not be ignored.)
Another likely difficulty is integration efforts. J has a variety of libraries, and has been connected to various other software systems, but the contrast with what has been accomplished for more popular languages is dramatic. My experience has indicated that J interfaces pretty readily, which I attribute to some good choices as to how to work with the outside world. (This includes DLLs, memory-mapped files, sockets, and text-file scripts.) Nevertheless, in a large project you'd be innovating much more often than usual when making connection to surrounding support software.
In my mind, the biggest factor would be whether I had an adequate team for undertaking the core product. This might realistically be a much smaller team than most "large" projects rely on. The array languages have been associated with significantly smaller teams than are typical for the field, which I consider to be among the advantages. If you have arranged to get the involvement of whatever colleagues you'll need, great. If not, don't commit to such a project until you do.
The next most important things would be knowing what you expect to obtain as the primary benefits of J, and ways to check that you're getting those benefits as you move through the project. I'm not the right person to advise how to do this, but I am sure you don't want to devote investment capital without keeping track of whether a high-risk choice like this is bearing out the anticipated rewards.
Paul Graham has written that some languages are relied on as secret advantages. J could definitely be such a language. Merely using J, however, would not be enough to assure that it gives the sort of advantages he had in mind. What you described as the appealing aspects of J suggest that such an advantage might be within reach for you. If you do choose to rely on it, it could be vital to have all the major participants have a strong, and sober, understanding why this language was chosen for that work.