8

Do you believe there is a gap between computer science research and software engineering problems? For example, do software engineers ever have to worry about "factoring and graph isomorphism" or some complex computer science problems if they have to....say build a shopping cart website? Probably not.

And if there is a disconnect between computer science and the engineers that build applications? Is that the way that engineering and science should exist? Will engineers dive through years and years of research papers to solve a particular problem that they have?

Edit-1: After thinking about it, general science probably has the same problem. I am sure there are top chemists working at companies like McDonald's and Taco Bell that are tasked with making a better, easier to manufacture burger.

berlinbrown2
  • 525
  • What gap? If you want tenure and grants then you worry about "factoring and graphi isomorphisms" as you put it and if you want to build a shopping cart site then you worry about cookies and session data. What gap is there to bridge? –  Apr 26 '11 at 23:09
  • You definitely should worry about graphs and such if you're building any kind of a workflow (including a shopping cart or anything equally mundane). Otherwise your workflow may become unusable and incomprehensible for both maintainers and your end users. – SK-logic Apr 17 '13 at 08:47

6 Answers6

10

In most software engineering positions, fundamental computer science problems rarely pop up, for one of two reasons:

  • They have been solved by the tools you use. Examples are parsing algorithms for the programming language you use by your compiler, scheduling algorithms for the applications you run by the operating system, query resolution in the database you use, etc.
  • They simply are not crucial to whatever you are trying to achieve. Not because they wouldn't help, but rather because the mundane automation task is much more important than the maximum optimized version.

The reason so many software engineers are building information systems that are trivial from a theoretical point of view, is simply because they are needed. The way our world is currently automated is probably at less than 0.01% of what could be achieved. So for the coming decades we're probably just going to be building mostly information systems and interfaces. Once we have those, some fundamental problems will start popping up.

These problems currently exist, for instance with regard to scalability, threading, etc. but they are simply an extremely small part of everything that needs to be done. So the reason companies are building relatively trivial information systems over and over is because (1) people need them and (2) it's much easier (and more lucrative) than solving fundamental problems.

Deckard
  • 3,427
  • 1
    " The way our world is currently automated is probably at less than 0.01% of what could be achieved." That is what I was thinking. It seems unfortunate that all of that advanced science, research, and genius may get lost because we are building applications that basically take data from one format and put it in a database. – berlinbrown2 Apr 26 '11 at 22:35
  • 3
    @Berlin, I think that's the problem frameworks are trying to address. Either way, we're moving in the right direction. Think about how much genius, etc., used to get wasted filing records in cabinets! – Ethel Evans Apr 27 '11 at 00:19
6

Do you believe there is a gap between computer science research and software engineering problems?

My experience is that commercial/practical software development lags behind academic research by 5 to 30+ years. One of the fastest timeframes from a breakthrough academic paper to a commercial shipping product was SQL. The paper was published in 1969, IBM and others spent a lot of time and effort trying to make viable products, and the first real commercially viable product was Relational Software - the company now called Oracle.

Functional languages were developed by researchers in the 1960s. How many are in common use today? Some. They're getting a lot more use these days than they did outside of the ivy-covered walls of universities. But it took three decades to do so.

Will engineers dive through years and years of research papers to solve a particular problem that they have?

Yes. I do it all the time. When I worked at a company that made storage area networks, many of the products starting to get shipped were described in research papers published 5-6 years earlier.

Another example involved a problem called "patient matching." Humans are good at looking at things like Chem. Dept. or Department of Chemistry and determining such things are identical. Most algorithms have a terrible time determining such things. I was working at a company that handled electronic drug prescriptions, lab reports and insurance claims. It would have been helpful to be able to (anonymously) be able to have long term data covering the efficacy and effectiveness of treatments for patients. Such a thing would have needed to depend on the ability to determine the closeness of strings. During the 1990s, most researchers in this area vanished into the Human Genome project, and most of their work disappeared off the web (with NDAs and intellectual property, everything these folks invented vanish from the web when they go to work for private industry). After 911, matching names became a "national security" issue (there are about 25 ways to spell Mohammed in English, and about a dozen ways to spell Osama) and many of the remainder vanished as well. So one inventor/company had a product that let you match people and relationships called "non obvious relation analyzer" which ended up vanishing into an add-on for DB2. You'll have to dig into papers a lot. Maybe not if you make shopping carts, but it is quite common to do so in other projects.

Thesis: Adaptive detection of approximately duplicate database records and the database integration approach to information discovery.
Library that implements some of the functions in the thesis.

Tangurena
  • 13,304
  • Good response and you connected an actual thesis to an application. I still argue that isn't the norm for most software engineering, development tasks out there. Like you said, I think software development may be years or decades behind research that is out there. And that is for interesting tasks like 'google search' or biometric applications. – berlinbrown2 Apr 26 '11 at 23:08
  • That is not true. For instance, recommender systems, market basket analysis, etc. all came out of fundamental computer science research using Bayes Theorem and this did not take 30+ years. – Mushy Apr 16 '13 at 19:29
4

Academic Computer Scientists are very good with the following:

  1. Analysis of algorithms
  2. Knowledge of Standard Data Structures.
  3. Automata theory

All of above are useful things for software engineering. In fact it would be indispensable to have at least one computer scientist in a software engineering team.

However, the way computer science is taught, and ABET accreditation rules exacerbate the problem (if it can be called a problem). The computer scientists do not have much knowledge of following key software engineering areas.

  1. RAM footfprint of a software and impact it may have on software performance.
  2. Maintainabilty engineering of software. They have no appreciation of the fact that 80% of software jobs are in maintaining software, which is the biggest component of software cost.
  3. How to work close to the hardware. This is required when you view entire system as a combination of hardware and software and you optimize the system to see as to how to optimize both for best functionality.
  4. Inventing new, fast, and reliable software testing, implementation, and maintenance procedures.
  5. Software documentation, client training and related documentation.
  6. System security engineering.

I can go on and on but I think I have made my point.

Software engineering today is a discipline of its own which borrows from computer science, but is driving the technology and human life today. You really need an Engineer's brain to excel in it. All computer scientists are not cut out to be great software engineers. Of course reverse will not be true either.

gnat
  • 21,213
  • 29
  • 113
  • 291
Satish Singhal
  • 49
  • what is "ABET"? – gnat Apr 16 '13 at 16:01
  • I disagree. Obtaining knowledge of software development is not some high, aloof thing as you make it out to be. I have a Masters in Computer Science and I do not lack knowledge om key software engineering areas. Most software engineers today are degreed and picking up on new knowledge is easy ... we have proven we can learn! – Mushy Apr 17 '13 at 14:05
2

I'd actually argue that to build a good shopping cart website, you absolutely need to employ difficult algorithms.

Let's say that you want to predict user behaviors based on past purchases. That's going to take a lot more than a+b=c to do effectively. How 'bout purchasing habits based on a number of different factors, such as age, gender, geographical location, etc?

In my own line of work, there's daily use of complex algorithms in rendering, AI, etc.

In short, if you think of a specific feature (i.e. just a shopping cart), then you're most likely thinking about poor implementations. Start thinking google or amazon implementations and I'm sure you'll start seeing where it would be useful (or required) to know or at least be familiar with complex algorithms.

Demian Brecht
  • 17,585
  • Yea, practical search and sorting algorithms are a part of computing. But I wouldn't call the basic sorting algorithms interesting problems in computer science research. I see AI including computer vision, genetic algorithms having more practical applications but there are other aspect of computer science research that seems far removed from software engineering. – berlinbrown2 Apr 26 '11 at 22:34
  • @berlin: I'd imagine that if you got a job at NASA or the NSA as a software engineer, you'd be faced with all kinds of comp sci research type algorithms. "Software Engineering" is a pretty vague term and can mean a wide variety of things depending on the context of the employer's field. – Demian Brecht Apr 26 '11 at 23:22
2

Problem solving software engineers have an enormous overlap with computer science research, and also mathematical & statistical research.

Designing a website isn't software engineering, even if you integrate some shopping cart code. It's designing.

Even 'coding' isn't necessarily software engineering - I know plenty of coders who wouldn't consider themselves anything approaching engineers. Code can be as simple as string manipulation or writing Excel formulae.

Obviously not all software engineering will overlap with scientific research (there are a lot of other responsibilities in the job), but I've read a lot of published papers in order to determine the optimal algorithm or approach to problems. These problems may arise only once a year (the rest of the time I'm writing UI validation or whatever) but that's the nature of my work environment.

Kirk Broadhurst
  • 4,209
1

Do you believe there is a gap between computer science research and software engineering problems?

No.

For example, do software engineers ever have to worry about "factoring and graph isomorphism" or some complex computer science problems if they have to....say build a shopping cart website? Probably not.

False. They use tools which depend on this being done correctly.

Indeed, all your friend-of-a-friend relationships in Facebook are a huge graph theory problem. Very complex. Very large. Very theoretical.

And if there is a disconnect between computer science and the engineers that build applications?

Yes. Some people build applications who are clearly unqualified. I've seen a lot of really shoddy stuff built by paid "professionals" that should have been doing something else, more useful with their time.

Is that the way that engineering and science should exist?

"Should" is meaningless. It's the way it does exist.

Will engineers dive through years and years of research papers to solve a particular problem that they have?

Yes. Often. That's why I subscribe to the ACM Digital Library. http://portal.acm.org/ It's essential for tackling problems that aren't trivial.

S.Lott
  • 45,390