10

I have an app-engine app that is deployed and running on the standard Python3 runtime. I am also able to run it locally using normal commands like flask run. But I am unable to run it the way I could run apps in the 2.7 runtime with dev_appserver.py. I am using the latest gcloud version, but running dev_appserver.py results in:

ERROR: Python 3 and later is not compatible with the Google Cloud SDK. Please use Python version 2.7.x.

I assumed we just aren't supposed to do it this way any more until it saw: https://cloud.google.com/appengine/docs/standard/python3/tools/local-devserver-command

Which starts with:

Python 3.7 Local Development Server Options

Usage:
dev_appserver.py [options] yaml_path [files...]

Is dev_appserver.py compatible with Python3 or not (maybe I somehow have an old version in my path)? If not, is there a new way to run apps locally that will adhere to the app.yaml(like treating static paths correctly) and give other niceties like the local data store that dev_appserver.py provided?

Community
  • 1
  • 1
Mark
  • 84,957
  • 6
  • 91
  • 136

3 Answers3

12

The dev_appserver.py support for Python 3 is still limited. From Running the local development server:

Note:

  • Running dev_appserver requires the presence of Python 2.7.12+ on your local machine.
  • The updated dev_appserver does not support development of Python 3 apps on Windows.

Which might be why it is not the recommended solution for local development (or at least not yet). From Running locally:

We do not recommend that you depend on tools like dev_appserver, the local development server provided with the Google Cloud SDK. However, if you are migrating an existing App Engine application from Python 2 to Python 3, we have updated dev_appserver to facilitate this process. For all other local development scenarios, we recommend standard Python testing approaches.

For example, you can usually run a Flask application with Flask's development server using:

python main.py

Django applications can be started using:

python manage.py runserver

To simulate a production App Engine environment, you can run the full Web Server Gateway Interface (WSGI) server locally. To do this, use the same command specified as entrypoint in your app.yaml, for example:

gunicorn -b :$PORT main:app
Dan Cornilescu
  • 38,757
  • 12
  • 59
  • 95
  • This was the piece I was missing. Thanks Dan. – Mark Oct 26 '18 at 18:09
  • 6
    The solution they are suggesting leaves much to be desired, first, what about the url handlers on app.yaml among other things? now we need to duplicate that on our python apps--and cannot test it before production. Second, where is this "updated" dev_appserver? or how would one get it? updating the gcloud yields the same error. Bad app engine. – maininformer Nov 27 '18 at 18:19
  • 3
    @plumSemPy This is my frustration too. In gen 1 GAE, almost all development and test could be done locally. in gen 2, one has to constantly deploy his App in order to test it, slowing down development a lot. I've decided to stick with Gen 1, and then if i need Client Libraries, to make a separate "API" App in Gen 2 GAE for my main Gen 1 App to call. Allows most development to be done locally still. – hamx0r Feb 15 '19 at 00:05
  • 4
    its really annoying and almost like Google don't want users on the App Engine standard platform anymore. all the good features on AppEngine have gone with the move to Paython3 so porting apps from 2.7 Python3 is basically a re-write and if we are re-writing applications we might as well look at other providers.... real shame. – Joe Bourne Mar 01 '19 at 01:19
  • @dan-cornilescu where is the "updated" dev_appserver? Anyway this: "We do not recommend that you depend on tools like dev_appserver, the local development server". Seems ironic – kip2 Mar 04 '19 at 08:19
  • 1
    @kip2 - in the current gcloud SDK (the older GAE SDK may not have it). – Dan Cornilescu Mar 04 '19 at 12:14
  • 2
    Migrating an app to python 3 is a nightmare. It really does seem like google was not prepared to launch this yet but had to due to the impending python 2 deprecation. Local development is harder because of the missing app server and the documentation and features are still miles apart from what you had on python 2, really frustrating. – Kreender May 12 '19 at 12:12
  • 3
    It is 2019-06-04 and the situation is still a problem. The biggest problem is inability to run the local version of google cloud store. I may be wrong about it. I just do not see the way to do it. – Michael Jun 04 '19 at 17:42
  • @Michael: did you try the [Datastore mode Emulator](https://cloud.google.com/datastore/docs/tools/datastore-emulator)? – Dan Cornilescu Jun 05 '19 at 02:18
  • @DanCornilescu, sorry no. It is a good tip. Looks nifty. Have you tried it out yourself? Does it work well on Windows? – Michael Jun 27 '19 at 10:55
  • @Michael I only tried it for this answer https://stackoverflow.com/a/50860147/4495081, actively I'm still using the dev server emulation. And on Linux, I'm not a windows fan ;) – Dan Cornilescu Jun 27 '19 at 11:09
  • @DanCornilescu, Ok, checked it out. Couple of things. One, the Cloud Datastore Emulator is actually slowly becoming the implementation of dev_appserver uses behind the scenes. You can start dev_appserver using CDE via a command line switch. If you do, you'll see console out of a message stating it is being slowly rolled out, and available for some people. Next I ran dev_appserver without the switch - and turns out I am one of those lucky to having it. The biggest problem so far with GCE is the lack of standard UI. Exploring UI options now – Michael Jun 28 '19 at 05:04
1

For windows 10 :

The solution was to install a linux subsystem for windows : https://docs.microsoft.com/en-us/windows/wsl/install-win10

I use the Ubuntu app from the windows store, and follow the installation for installing the gcloud component on linux inside the ubuntu terminal: https://https://docs.microsoft.com/en-us/windows/wsl/install-win10cloud.google.com/sdk/docs/#deb

Inside the Ubuntu terminal it is possible to access the windows file : the C drive would be /mnt/c. Getting back to my work folder, it is possible to start the web-app using the dev_appserver command.

Using a navigator from windows we have access to the web-app as normal using localhost:8080.

The development can still be done using an IDE in windows, running the server in Ubuntu.

Stephane Duteriez
  • 934
  • 8
  • 8
0

Adding an update in 2022

  1. From the link Running the local development server mentioned by @dan-cornilescu in his response, they have changed the verbiage and removed the text that says ...we do not recommend that you depend on tools like dev_appserver.... . Instead the text now says

We recommend that you use standard Python tools, such as virtualenv to create isolated environments...

  1. It also looks like they have enhanced dev_appserver.py to work with Python 3. Running the local development server now says

To run dev_appserver with a Python 3 interpreter, you must specify the --runtime_python_path=[PATH_TO_PYTHON3_BINARY] flag.

  1. I don't know why their documentation gives the impression that the advantage of using standard python tools over dev_appserver.py is to be able to have virtual env given the fact that when you run your Python3 App with dev_appserver.py, it will first create a temp folder, create a virtual environment in that folder and install the contents of your requirements.txt file there.

  2. The downside with using dev_appserver.py is that each time you start your App, it will create a new virtual env and reinstall the contents of your requirements.txt. This means that while you're still troubleshooting your App (which might involve multiple restarts), you will end up with lots of folders in your tmp directory and these are not deleted when the app is shut down. There is also the time it takes to install the requirements.

  3. If you start the Cloud Datastore Emulator and set the flag for dev_appserver.py to use it, then you'll be able to view your data in the same UI that you're used to for Python 3 projects (@michael this should answer your question about UIs). Below is a screen shot

    enter image description here

  4. There are also 3rd party UIs from - https://github.com/GabiAxel/google-cloud-gui, https://www.npmjs.com/package/google-cloud-gui

  5. Our App, NoCommandLine also has a UI for datastore

    enter image description here

NoCommandLine
  • 2,297
  • 1
  • 3
  • 8