How to use pthread_setaffinity_np

Question

I am working on a performance critical system that has a lot of logging. I am planning to do my major computation in a thread that sticks to one core, and logs in another thread that sticks to another core. How do I achieve that in pthread_setaffinity_np() ?

I imagine that my code would look like

void log_something(const string& st) {
  pthread_setaffinity_np(pthread_self(),sizeof(cpuset),&cpuset);
  //LOG string st; 
}

int main() {
   while (true) {
      // do some computation
      log_something(something)
   }
}

My question is, when I call pthread_setaffinity_np() in log_something(), would it do the logging in the cpu I specified, and while doing the logging, return to the while loop of main() and continue the computation?

Or it will switch the entire program to that cpu set, and will only return to main method after logging?

Thanks!

[EDITED] I use logging as an example but my practical problem is more complicated than that. For example, I might have to update a dynamic parameter every minute, and while I am updating the parameter, I still want to continue my computation in the main() method based on the old parameter (i.e. I just can't stop my main computation for parameter updating). Therefore the process of updating the parameters might have to be migrated to another thread that sticks to another core. So I am looking for a generic solution of separating the computation, not just an efficient logger.

Sorry for the confusion.

Is there a particular reason why you think you need to force threads to stick to particular cores, instead of spinning up a few threads or using some existing threading library and letting it manage the threads? — dsolimano, Nov 16 '11 at 15:18
@dsolimano: there are plenty of good reasons for doing this; in particular, allocating cores so that threads that share data also share caches, and making sure interrupts are always handled on different cores to critical computations. I'm not aware of any threading library that will optimise such things automatically. — Mike Seymour, Nov 16 '11 at 15:28
@Mike, I'm aware that there are plenty of good reasons to do it. But the question is, does our friend usfish need to do it for any of those reasons, or is he just working with the wrong tools because he doesn't know there are other ones better suited to his needs. — dsolimano, Nov 16 '11 at 15:30
A logging thread does not do much at all. Hang on a queue, get a log object, act on it, probably write the contents to a file, go back to queue. Why take the effort of sticking this to a core? I suspect your attempts to improve performance by second-guessing the OS scheduler will not be as effective as spending the same effort on, say optimizing your calculation algorithm, paralleling it or just buying faster memory/CPU. — Martin James, Nov 16 '11 at 15:43
@MikeSeymour - there are maybe gains to be had, but are they worth it compared with spending the time on other optimizations? Even if the OP manages to force all the ISR and interrupt tasks away from one core and lock the 'computation' to it, would this not be hardware and OS specific, ie need individual setups and regression tests on all OS? To me, if all other reasonable optimizations have been done the app performance is still insufficient, the box needs better/faster hardware. — Martin James, Nov 16 '11 at 16:04

score 1 · Answer 1 · edited May 23 '17 at 11:47

When you call pthread_setaffinity_np, it sets that thread to execute on the specified cpuset forever. It is not somehow method-scoped. From the documentation,

The pthread_setaffinity_np() function sets the CPU affinity mask of the thread thread to the CPU set pointed to by cpuset. If the call is successful, and the thread is not currently running on one of the CPUs in cpuset, then it is migrated to one of those CPUs.

It sounds like you want to spin off a separate thread for logging in your main function, then call pthread_setaffinity_np with different cpuset arguments in the main function and the logging thread. Then you need some way to communicate from the computational threads to the logging threads, perhaps a lock and a queue, perhaps something more complicated. That could be a lot of work.

You probably want to look at this SO question which talks about efficient C++ logging libraries and choose one of those, as I suspect the library authors have already done the hard work.

score 1 · Accepted Answer · answered Nov 16 '11 at 15:17

You'll need two separate threads, and each will call pthread_setaffinity_np when it starts up.

If log_something is called from the computation thread, and logging is intended to be done on the logging thread, then it will have to place the message on a queue for the logging thread to deal with, rather than doing the logging itself. The logging thread will poll that queue, and log any messages it finds there.

How to use pthread_setaffinity_np

2 Answers2