Help us collect modern Python NumPy code solutions(github.com)

Help us collect modern Python NumPy code solutions

https://github.com/Onelinerhub/onelinerhub/labels/python-numpy

40 comments

For almost all issues, you can simply search for site:stackoverflow.com + the title of the issue and get a larger variety of solutions ranked by upvotes, so this repository seems strictly worse.

Is there any advantage that this repository brings over just searching for the question on StackOverflow?

charlieyu14년 전

A lot of times the best solution is buried in the third or fourth solution in the 5th or 6th search result. I can’t even find the solution that I know I have googled before.

JoBrad4년 전

I didn’t review all of the code, but the examples I saw seemed to have some basic explanations with them. Additionally, they have samples for a lot of different technologies. A curated repo seems much nicer than scoring several StackOverflow answers. Depends on the code quality, of course.

johndough4년 전

The code quality seems questionable. A few examples:

Repository claims modern code, but uses Python 2 (end of life January 1, 2020) https://github.com/Onelinerhub/onelinerhub/blob/main/python/... Copied from https://wiki.python.org/moin/Powerful%20Python%20One-Liners

Incorrect indentation causing unnecessarily large memory usage https://github.com/Onelinerhub/onelinerhub/blob/main/python/...

Undefined behavior for i = 31 https://github.com/Onelinerhub/onelinerhub/blob/main/c/bits_...

Printing of uninitialized memory https://github.com/Onelinerhub/onelinerhub/blob/main/c/set_v...

Undefined behavior https://github.com/Onelinerhub/onelinerhub/blob/main/c/swap_...

Beltiras4년 전

Fixed some of those. Simple case of just pushing the edit button and sending a PR.

johndough4년 전

The suggested solution https://github.com/Onelinerhub/onelinerhub/blob/34467e427cc6... still runs out of memory with many files.

I believe the original intention behind using mode="a" was to append to the output file while reading the input files at the same time. This way, there is no need for an ever-growing string array.

But there are still many other issues like using default platform string encoding instead of detecting it properly or at least using utf-8, checking for ".txt" anywhere in the path instead of at the end, and not closing the input files with a context manager like the output file, which suggests that this code is just pierced together from various sources.

A robust solution would require many more lines.

hermitdev4년 전

For bonus points: how about properly handling UTF-8 byte order marks (BOM), too? The way this code is written if the files contain a BOM, you're going to pass the BOM right into the destination file. Fine for the first file, not so fine for any other file. Dropping a BOM mark in the middle of a file isn't going to result in valid UTF-8...

mynameismon4년 전

Another memory issue (although unlikely) could also be the file itself being larger than the RAM size, which can be avoided by chunking the textfiles.

danbruc4년 전

I looked at the C# ones - the code is not great and neither are the explanations, both show a lack of experience in C# and .NET.

  How to reverse the characters in a string

  String.Join("",("1234Simple").Select(c=>c).Reverse());

  - String.Join("", - concatenate one or more characters in a single list or array, by using empty "" string literal separator
  - .Select(c=>c) - select each and every other character in a string into an array of characters
  - .Reverse() - reverse each and every one elements in an array or list*

Calling Select(c => c) does nothing and can be omitted. The use of String.Join() feels wrong, it is just used to concatenate the characters and not to join them with some separator in between. Using the String(Char[]) constructor seems more appropriate.

  new String("1234Simple".Reverse().ToArray())

But that is still bad code, it will rip apart and reverse grapheme clusters, a proper implementation should use System.Globalization.StringInfo. Not sure if it would still fit into one line.

Also the explanation are not very accurate and show a lack of understanding of the .NET collection type architecture and how array, list and enumeration types fit together.

  How to reverse the words in a phrase

  ("This tests that").Split(' ').Aggregate((a,b) => b + " " + a);

  - .Split(' ') - splits a string by using a space ' ' separator
  - .Aggregate - gather every one other element a, in an element list to a target element b
  - (a,b) => b + " " + a - arrow function taking input arguments (a,b) used to concatenate b gathered character argument in the list in reverse with " " space separator to the targeted character argument a

Here I would say IEnumerable<T>.Reverse() paired with String.Join() is the better solution as it is much easier to unterstand than using IEnumerable<T>.Aggregate().

  String.Join(" ", "This tests that".Split(' ').Reverse())

Now nobody has to figure out that (a, b) are the accumulated result and the current list element in this order which is even with the explanation not to obvious - better variable names could really help here.

  How to retrieve numeric fibonacci list up to 10

  List<long> Fib = new List<long>(); Enumerable.Range(0,10).ToList().ForEach(n => (Fib).Add(n <= 1 ? 1 : Fib[n-2] + Fib[n-1]));

  - new List(); - create an empty list of large integers
  - Enumerable.Range(0,10) - get an enumerable list populated with integers from 0 to 10
  - .ToList() - convert the enumerable list to a standard list
  - .ForEach - apply a specific arrow function for each and every one item in the list
  - n => (Fib).Add - arrow function used to add an item n to enumerable list Fib based on a condition for every one item in the list
  - n <= 1 ? 1 : Fib[n-2] + Fib[n-1] - check if n less or equal to 1 then return 1, if n is larger then return addition of elements [n-2] and [n-1] in Fib list for every n

This is really abusing LINQ to express a for loop - create an IEnumerbale<T>, materialize it as a List<T> only to be able to use List<T>.ForEach() which is - for better or worse - not available for IEnumerable<T> and then manipulate a different list created with a second statement in the same line. If you desperately want a one line solution for Fibonacci numbers, maybe consider using a closed-form expression, the recursive definition just doesn't work well in C# and a single line. This will however run into precision issues for big enough numbers while working with integers will work fine until Int64 overflows.

  Enumerable.Range(0, 10).Select(n => (Int64)(0.5 + Math.Pow((1 + Math.Sqrt(5)) / 2, n) / Math.Sqrt(5)))

Or with using static System.Math even shorter and more readable.

  Enumerable.Range(0, 10).Select(n => (Int64)(0.5 + Pow((1 + Sqrt(5)) / 2, n) / Sqrt(5)))

The explanation has again issues with the collection types, Enumerable.Range(0, 10) generates 10 values starting from 0, i.e. 0 to 9, but maybe they wanted to say from 0 (inclusive) to 10 (exclusive). Also arrow functions are called lambda functions or lambda expressions in C#.

xnorswap4년 전

Indeed, their example is terrible, because it's also the kind of 'toy' problem that just doesn't have good real world uses too.

How often do you want to reverse a string.

I think your solution is fine and isn't "bad code" at all. Certainly a real explanation warrants talking about the IEnumerable interface.

I'd also consider the simpler (and ever so slightly better performing):

    var buffer = "1234simple".ToArray();
    Array.Reverse(buffer);
    new string(buffer);

edit: The following is dramatically (x10) even better performing:

    var buffer = "1234simple".ToCharArray();
    Array.Reverse(buffer);
    new string(buffer);

But again with the caveat of that not respecting unicode sequences.

But to re-iterate and to add to your criticism, I can't understand the use-case of needing to do this outside of toy problems.

kingaillas4년 전

> I can't understand the use-case of needing to do this outside of toy problems.

Not sure, not my domain, but perhaps bioinformatics has the equivalent of the convolution operation on sequences?

EDIT: there is the "reversal" operation. Peeking ahead from the Rosalind problems, I'm not quite up to Reversal Distance (https://rosalind.info/problems/rear/).

danbruc4년 전

But would they represent their data as character strings or would they pack four nucleotides with two bits each into one byte? Or would they be working on the amino acid level or something else all together? And would they really physically reverse potentially very long strings or just use a decreasing index into the original string? I am sure reversing strings has some use cases, maybe even in bioinformatics, but I would agree with the other comment, it seems a pretty niche operation, otherwise there would probably be a String.Reverse() method.

jmmcd4년 전

The README explains that they want a single curated answer, not a list of 15 answers with 25 comments. This does seem worthwhile, at least for some types of questions.

johndough4년 전

Isn't the "curated" answer on Stack Overflow the answer with most votes?

Sure, sometimes the top voted answer is not the answer to my particular problem at hand, but if it isn't, I'd rather have more alternative answers than nothing.

hackernewds4년 전

To a beginner they're not really able to make that determination, neither would have the patience to stay interested checking SO hundreds of times

danuker4년 전

Even better, search the stackoverflow.com site directly. It seems Google no longer indexes everything (if it ever did).

ledauphin4년 전

I would love to see something like this for `pandas`. Also SQLAlchemy. EDIT: apparently there's a very limited set of oneliners for `pandas` already.

Why do so few libraries seem to understand that good documentation starts by providing meaningful examples of simple tasks with semantic labels that can be found via search? Reference-style documentation assumes that you already know the name of the thing you're looking for (you don't), or that you have time to read for several days and build up an entire mental model of the system in one go before getting anything productive done (also no).

na854년 전

The worst is when some project or library is extolled as being extensively documented and you find out it's just doxygen autogens.

Awful.

muggermuch4년 전

+1 absolutely - it's been one of my pet peeves. There really needs to be some thought put into "doc UX".

omarhaneef4년 전

Just had a quick look at some of the responders who collectively say:

1 What is the use of this curated list over just accepting the popularly voted answer?

2 I found some errors in the default answer and here is a detailed explanation of what was wrong with the answer.

And I hope they can see that #2 shows the issues with #1.

To put it another way if I were looking for the best solution (which I often am), and I had a choice between a one line stack overflow answer and one of these detailed solutions, I would pick the detailed, curated solution every time.

edit: And the best stack overflow answers are often detailed curated solutions, so I am not saying they don't wind their way up there.

denisrosset4년 전

I'm starting to add type annotations to numerical code, and I wonder how to handle the disconnect between `float` (which is how floating-point literals are typed) and concrete NumPy types such as `numpy.float64` (which is the scalar type I get out of NumPy arrays).

I ended up defining an alias:

Float = Union[float, numpy.float64]

but I wonder if there is a more legitimate way to handle thiss.

m00dy4년 전

#write a function that zips numpy arrays

import numpy as np

   def zip_arrays(arrays):

       return np.array([np.concatenate(arr) for arr in zip(*arrays)])

PartiallyTyped4년 전

>>> arrays = [np.arange(1000)]*10

>>> zip_arrays(arrays)

Traceback (most recent call last):

  File "<stdin>", line 1, in <module>

  File "<stdin>", line 2, in zip_arrays2

  File "<stdin>", line 2, in <listcomp>

  File "<__array_function__ internals>", line 180, in concatenate

ValueError: zero-dimensional arrays cannot be concatenated

Try this:

def zip_arrays(arrays):

    return np.concatenate([array[:l,...,None] for l in [min(array.shape[0] for array in arrays)] for array in arrays], axis=-1)

Or if we are trying to be more tidy instead of a one liner

def zip_arrays(arrays):

    l = min(a.shape[0] for a in arrays)

    arrays = [a[:l, ..., None] for a in arrays]

    return np.concatenate(arrays)

johndough4년 전

I think a more Pythonic solution would be

    np.array(arrays).T

or maybe

    np.copy(arrays).T

for one less character. The shortest solution is probably

    np.c_[arrays].T

but likely causes a Google search on first read.

PartiallyTyped4년 전

That does not zip though. In fact, it creates a 1-dimensional array of objects because the arrays do not have the same number of items.

     >>> a = [np.arange(i) for i in range(1, 11)]

     >>> np.array(a).T

        array([array([0]), array([0, 1]), array([0, 1, 2]), array([0, 1, 2, 3]),

        array([0, 1, 2, 3, 4]), array([0, 1, 2, 3, 4, 5]),

        array([0, 1, 2, 3, 4, 5, 6]), array([0, 1, 2, 3, 4, 5, 6, 7]),

        array([0, 1, 2, 3, 4, 5, 6, 7, 8]),

        array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])], dtype=object)

In fact, if we are to zip them, it should yield:

>>> list(zip(*a))

[(0, 0, 0, 0, 0, 0, 0, 0, 0, 0)]

zip_arrays otoh does give the same result.

>>> zip_arrays(a)

array([[0, 0, 0, 0, 0, 0, 0, 0, 0, 0]])

johndough4년 전

You are completely right. Nevertheless, when I zip over arrays of different length, it is usually a bug instead of being intentional.

A code-golfed version for different array lengths:

    np.c_[[a[:min(map(len,arrays))].T for a in arrays]].T

The trick to get a local variable within a list comprehension statement with

    for l in [foo()]

from your zip_array function is very cool by the way, but I sacrificed it in favor of shorter code.

PartiallyTyped4년 전

I like your your solution, I think using `a[0:min(map(len,arrays))]` (ie with 0) is faster because it returns a slice and not a copy of the array.

I am glad you liked the trick!

kzrdude4년 전

I'm used to numpy and I can't understand np.c_ even with the docs. It says "Translates slice objects to concatenation along the second axis." but the examples pass in a list of arrays and use no slice objects (the builtin slice, I'm assuming)?

If there was an example using just `np.c_[1:5, 11:15]` it would make sense. But clearly np.c_'s meaning has been extended beyond slices in some way. (Maybe I'm coming closer to understanding, but still miffed about the misleading doc.)

m00dy4년 전

well, I used the code from Copilot :)

kwertzzz4년 전

Would it be useful to add a Julia section?

infinityio4년 전

Just had a quick look at a few of the python examples:

- it might be neat, but advocating for `eval(input())` [0] might not be the safest solution for this problem, especially without explaining the dangers of `eval` (assuming this site is partially aimed at beginners?)

- for an article titled 'how to terminate a script', the suggested method (`quit()`) [1] is specifically described in the official python docs [2] as code that "should not be used in programs".

[0]: https://onelinerhub.com/python/calculator [1]: https://onelinerhub.com/python/how-to-terminate-script [2]: https://docs.python.org/3/library/constants.html#constants-a...

kortex4년 전

I agree eval is pretty gross and folks should not be encouraging its use since it can lead to bad habits.

Quit/exit are pretty innocuous though. I actually just thought exit was an alias for sys.exit. I've never had a problem using plain exit() in programs.

hoosieree4년 전

I learned J before NumPy and I'm glad I did, because it's much easier to see and learn the algorithms when they're short symbols.

Compare

    np.sqrt(sum(np.square(x)))

versus

    %: +/ *: x

So if you can translate one of the array languages to NumPy, you can tap into the wealth of "idioms" collected over the years.

For example: J Phrases[0] (organized by category), BQNcrate[1] and APLcart[2] (searchable).

[0]: https://www.jsoftware.com/help/phrases/sums_means.htm [1]: https://mlochbaum.github.io/bqncrate [2]: https://aplcart.info/

johndough4년 전

I think a fairer comparison would be

    np.linalg.norm(x)

or even

    norm(x)

if you

    import numpy.linalg.norm as norm

beforehand. This comes with the advantage that you can look up the function by name to know what it does instead of having to look up chains of symbols on several websites.

RodgerTheGreat4년 전

In practice, if you're reading a J expression which contains unfamiliar primitives, there's only one place you need to look: https://code.jsoftware.com/wiki/NuVoc

Idioms are composed from primitives, but many of them are shorter than a descriptive name would be, and since the set of primitives is small (compared to the standard library in most languages), it doesn't take that long to memorize most or all of the symbols.

kzrdude4년 전

Interesting suggestion since

    %: +/ *: x

Is impossible to read of course without learning the J language's main constructs. While the python function application is easier to guess from other languages.

hoosieree4년 전

I know English, so I'll just guess at what those non-English people are saying. What could go wrong?

kzrdude4년 전

I know spanish so I understand some french sentences. It's actually quite easy to see what I don't understand. It doesn't usually go wrong.

bdrum4년 전

I've added a comment to the issue How to plot a data frame https://github.com/Onelinerhub/onelinerhub/issues/1378#issue...

Actually concerning visualization tasks I don't see a sense in one line solution because as I think you should understand how you want to represent your data and this has strong influence to calling signatures.

TrackerFF4년 전

Some of these problems don't really have much information / specs.

"Generate random array" (python-numpy)"

Ok - but what dimension? size/length? what range? sample from what distribution? only unique values? etc.

Many of these problems have "native" one-liner numpy solutions which are trivial to find (basically just read the documentation).