2

I'm moving a Magento site to AWS, there will be 2 EC2 frontends in the new setups. I'm wondering if there are any standard approaches to managing things like the /var and /media directories so far as uploads?

As far as I know, the easiest way to handle this is to map those directories onto S3 buckets. Some applications also have in-built functionality to use S3, such that the aforementioned is not even necessary - but I'm guessing it would be too hard or even impossible to make a Magento extension to achieve the same effect?

Has anyone done this sort of thing before, and if so, what has been your approach? What are the pitfalls?

Thanks

SCM
  • 91
  • 1
  • 3

3 Answers3

3

TL;DR

NFS for media, Redis for sessions and cache.

If you are asking this question, part of me feels like advising you that you shouldn't attempt this endeavour alone.

When running a multi server configuration, you have a huge number of things to factor in.

  • Server application configuration management (ie. Chef/Puppet/Ansible), so that all machines in the cluster are exactly the same
  • Fault tolerance and management, so that nodes are healthchecked and removed from the load balancer when failed
  • Distribution of single server activities, ie. on which machine your crons run on
  • Deployment and management of codebase, so your Magento file structure is identical on all machines
  • Log aggregation and management (for centralised overview)
  • Service management, instrumentation and data collection (ie. graphing) of all respective elements from a single location

You need to give thought to all of these aspects, merely managing sessions/media is a tiny fraction of running a complex multi server configuration. When you multiply your server count, you multiply your points of failure, so you need to adequately plan for this.

Ie. You can't just run your NFS master from a single web server, because when that fails, your other machine will fail too. So you need to consider your high availability requirements from the start (ie. planning for failure).

If you want a reliable cluster, you'll need,

  • 2 Load balancers
  • 2 Web servers
  • 2 NFS stores
  • 2 Cache backends
  • 2 Database servers
  • 2 Monitoring servers

Ie. Two of everything.

Then configure high availability of each service layer, with full real time replication of data.

Save yourself the complexity, the risk, the confusion and just run a single web server and scale vertically until such point you hit a ceiling. Then at that point, you can assume your business is large enough to warrant using a proper managed service anyway. Learning to build a cluster at the cost of your own business is not wise.

Community
  • 1
choco-loo
  • 962
  • 6
  • 11
  • That sort of server management is not being done by me, I'm only responsible for the application side, as well as setting up the LAMP stack components. The infrastructure side of it is also not decided by me. – SCM Nov 23 '14 at 20:42
0

You need to map the media folder as an nfs share otherwise the images don't work, var is best to leave local for performance. You need Redis or Memcache for sessions.

serpyre
  • 101
  • 1
0

The simplest way to handle this is to setup one server as the master and set your Admin URL To this server.

For example: admin.domain.com -> 10.0.0.1 (ip is only an example)

Your Load balancer would go to -> www.domain.com

You can use Lysnc to push out any changes from 10.0.0.1 to each of your individual servers. You can still use a CDN and you will have to centralize your sessions and cache. I Would recommend using REDIS.

The also gives you the added benefit of testing your application on code pushes by turning off lsync and testing only the admin server by setting you local host file.

Lsync also allows you to create exclude lists so you don't have to sync the var/ folder etc

All your uploads through admin or sFTP or SCP will go to the 10.0.0.1 server and these changes will get pushed out to the slave servers on the network.

brentwpeterson
  • 6,104
  • 7
  • 43
  • 81