Access Django models with scrapy: defining path to Django project

Access Django models with scrapy: defining path to Django project

I'm very new to Python and Django. I'm currently exploring using Scrapy to
scrape sites and save data to the Django database. My goal is to run a
spider based on domain given by a user.
I've written a spider that extracts the data I need and store it correctly
in a json file when calling
scrapy crawl spider -o items.json -t json
As described in the scrapy tutorial.
My goal is now to get the spider to succesfully to save data to the Django
database, and then work on getting the spider to run based on user input.
I'm aware that various posts exists on this subject, such as these: link 1
link 2 link 3
But having spend more than 8 hours on trying to get this to work, I'm
assuming i'm not the only one still facing issues with this. I'll therefor
try and gather all the knowledge i've gotten so far in this post, as well
a hopefully posting a working solution at a later point. Because of this,
this post is rather long.
It appears to me that there is two different solutions to saving data to
the Django database from Scrapy. One is to use DjangoItem, another is to
to import the models directly(as done here).
I'm not completely aware of the advantages and disadvantages of these two,
but it seems like the difference is simply the using DjangoItem is just
more convenient and shorter.
What i've done:
I've added:
def setup_django_env(path):
import imp, os
from django.core.management import setup_environ
f, filename, desc = imp.find_module('settings', [path])
project = imp.load_module('settings', f, filename, desc)
setup_environ(project)
setup_django_env('/Users/Anders/DjangoTraining/wsgi/')
Error i'm getting is:
ImportError: No module named settings
I'm thinking i'm defining the path to my Django project in a wrong way?
I've also tried the following:
setup_django_env('../../')
How do I define the path to my Django project correctly? (if that is the
issue)