This link presents a good example along with screenshots that would help you compile your basic openCV code.
Basically, whichever way you choose to compile, you would have to mention the path to openCV files.
When you use terminal to compile you will use an option similar to the one below:
g++ -I/usr/local/include/opencv -I/usr/local/include/opencv2 -L/usr/local/lib/ -g -o binary main.cpp -lopencv_core -lopencv_imgproc -lopencv_highgui -lopencv_ml -lopencv_video -lopencv_features2d -lopencv_calib3d -lopencv_objdetect -lopencv_contrib -lopencv_legacy -lopencv_stitching
If there are multiple files that you intend to compile, you can include them in place of main.cpp in the above command.
The locations after -I provide location for the header files. -o option is used to set the name of the output binary file. You further explore the significance of other options in g++ command.
You would only have to modify the path in the above command while using it for your applications.
If you have a larger application, then writing a makefile or using cmake would make sense. Use of IDEs like Eclipse or CodeBlocks can also simplify things to some extent.
Further expanding your question would give a better understanding of your requirements.