Dealing With Large-scale Permutations

james south - I checked it out a bit and it doesn't appear that I can get it to work the way it should. For example, my adapted Pythonic version of it with an input list with a length of 7 and passing in 7 as the integer argument, when finished running, will return a list with the expected length (5040), but it appears to have duplicates (1148 of them) when running this against the list, giving only 3892 unique permutations.

dups = set(tuple(x) for x in PERMUTATIONS if PERMUTATIONS.count(x)>1)

And if I instead use a check before appending to the list of permutations, I am left with a list that is 1452 unique permutations.

if tuple(A) not in PERMUTATIONS:

I'm pretty sure I "converted" it over properly but I could be mistaken since I wasn't able to decipher parts of it and faked my way through it. I probably botched it somehow that I can't sort out yet, or the formula on Wikipedia is flawed but, either way, it doesn't tackle the bigger problem which dealing with larger sets of permutations. Even with my flawed interpretation of the formula, once the permutation count gets too high, it will crash. I'm guessing it has something to do with running out of memory, but I'm no expert (obviously)

james south - I'm not entirely concerned with my flawed implementation of that article's algorithm, since it does seem to generate the expected count of permutations at lower levels. Even if it (my version) flawed and generates duplicates, this is something I could hash out later down the line. My main problem with it is that it performs much like Python's itertools, and several other attempts I've made at dealing with large-scale permutations. Once the permutation count reaches something around 350k+ permutations, all methods I have tried wind up hanging or crashing. I believe this is probably running out of memory or something, as I've read around online about many itertools implementations that have similar behavior, but I've yet to find a suitable solution. That's when I started to read up on generators and data streams but, as I said before, I haven't found a decent supply of understandable information on the topics to actually implement the principles myself yet.

To be perfectly honest, I'm not even sure what the best approach for dealing with large scale permutations would be. I assume that since I don't need all the data at once, it would be in the interest of performance to generate only a small series of permutations as I described in my original post (supplying a starting point and generating a set number of permutations), work with those permutations and then use the last permutation to seed the next series. To me, this sounds like something that could be done with a generator fairly easily, but I'm struggling with my understanding of them. I'm also not sure how I would implement any permutation solution (be it itertools, the one from your link, or any of the other solutions I've found that work in small scale) in a way that would work with a generator, or that would only return a smaller sample. Most of the permutation solutions I've found operate on a 1:1 basis where the permutations you get out are the same length as the iterable that's fed in. I'd like to be able to choose sample sizes.

I think I might just try to find an alternative to permutations, I dunno.

That's rather interesting to me, to be honest. Any online repl I've used, my server, and my personal laptop all refuse to go above a full 9 permutation using a single line itertools.permutations() implementation. If I implement my own algorithms it takes significantly longer, but again, I can't pass anything larger than a 9 item permutation, on any of the above devices. I sort of expect that from an online repl, but I must have something set up wrong on my personal devices that's preventing me from going higher. Perhaps there is a safety in Python installs that I am unaware of?