#StackBounty: #python #spacy invalid index to scalar variable on spacy array enumerate

Bounty: 50

I’m trying to run the following sentence compression locally: https://github.com/zhaohengyang/Generate-Parallel-Data-for-Sentence-Compression

So I copied the files and installed all dependencies with conda. I have made minor modifications such as reading the data from url instead of local disk and placed his parallel_data_gen.py bundled in my single py file.

However when I run it I get:

Spacy library couldn’t parse sentence into a tree. Please ignore this sentence pair

----------------59-------------------
reducing sentence: This year the Venezuelan government plans to continue its pace of land expropriations in order to move towards what it terms ``agrarian socialism''.
reducing headline: Venezuelan government to continue pace of land expropriations for ``agrarian socialism''
Traceback (most recent call last):
  File "/home/user/dev/projects/python-snippets/zhaohengyang/sentence-compression.py", line 701, in <module>
    reduce_sentence(sample)
  File "/home/user/dev/projects/python-snippets/zhaohengyang/sentence-compression.py", line 641, in reduce_sentence
    sentence_info = parse_info(sentence)
  File "/home/user/dev/projects/python-snippets/zhaohengyang/sentence-compression.py", line 616, in parse_info
    heads = [index + item[0] for index, item in enumerate(doc.to_array([HEAD]))]
IndexError: invalid index to scalar variable.

I’m not sure how to fix that as i’m a rather newbie python user.

Here is the full code i’m running to reproduce the problem: https://gist.github.com/avidanyum/3edfbc96ea22807445ab5307830d41db

The internal snippet that fails:

def parse_info(sentence):
    doc = nlp(sentence)

    heads = [index + item[0] for index, item in enumerate(doc.to_array([HEAD]))]

and here is now I loaded the nlp:

import spacy
print('if you didnt run: python -m spacy download en')
import spacy.lang.en
nlp = spacy.load('en')

More info about my env:

/home/user/home/user/dev/anaconda3/envs/pymachine/bin/python --version
Python 2.7.15 :: Anaconda, Inc.


Get this bounty!!!

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.