Author xtreak
Recipients martin.panter, orange, serhiy.storchaka, xiang.zhang, xtreak
Date 2019-03-15.06:15:37
SpamBayes Score -1.0
Marked as misclassified Yes
Message-id <1552630537.14.0.0284224358638.issue30458@roundup.psfhosted.org>
In-reply-to
Content
See also https://bugs.python.org/issue36276 for a similar report. I think it's better to raise an error instead of encoding CRLF characters in URL similar to headers.

I feel either of the issue and more preferably issue36276 closed as a duplicate of this one. Copy of msg337968 with reference to details about similar report in golang : 

For reference an exact report on golang repo : https://github.com/golang/go/issues/30794 . This seemed to have been fixed in latest golang release 1.12 and commit https://github.com/golang/go/commit/829c5df58694b3345cb5ea41206783c8ccf5c3ca . The commit introduces a check for CTL characters and throws an error for URLs something similar to Python does for headers now at bf3e1c9b80e9.

func isCTL(r rune) bool {
	return r < ' ' || 0x7f <= r && r <= 0x9f
}

if strings.IndexFunc(ruri, isCTL) != -1 {
	return errors.New("net/http: can't write control character in Request.URL")
}

So below program used to work before go 1.12 setting a key on Redis but now it throws error : 

package main

import "fmt"
import "net/http"

func main() {
	resp, err := http.Get("http://127.0.0.1:6379?\r\nSET test failure12\r\n:8080/test/?test=a")
	fmt.Println(resp)
	fmt.Println(err)
}


➜  go version
go version go1.12 darwin/amd64
➜  go run urllib_vulnerability.go
<nil>
parse http://127.0.0.1:6379?
SET test failure12
:8080/test/?test=a: net/url: invalid control character in URL

Looking more into the commit there seemed to be a solution towards escaping characters with https://github.com/golang/go/issues/22907 . The fix seemed to have broke Google's internal tests [0] and hence reverted to have the above commit where only CTL characters were checked and raises an error. I think this is a tricky bug upon reading code reviews in the golang repo that has around 2-3 reports with a fix committed to be reverted later for a more conservative fix and the issue was reopened to target go 1.13 .

Thanks a lot for the report @ragdoll.guo

[0] https://go-review.googlesource.com/c/go/+/159157/2#message-39c6be13a192bf760f6318ac641b432a6ab8fdc8
History
Date User Action Args
2019-03-15 06:15:37xtreaksetrecipients: + xtreak, martin.panter, serhiy.storchaka, xiang.zhang, orange
2019-03-15 06:15:37xtreaksetmessageid: <1552630537.14.0.0284224358638.issue30458@roundup.psfhosted.org>
2019-03-15 06:15:37xtreaklinkissue30458 messages
2019-03-15 06:15:37xtreakcreate